mirror of
https://github.com/whyour/qinglong.git
synced 2026-09-20 16:07:11 +08:00
feat(ql3): add profile-aware run log reads
This commit is contained in:
whyour
@@ -11,6 +11,22 @@
|
||||
|
||||
最新增量证据（2026-08-12）：
|
||||
|
||||
- D-289/ADR-0377（已接受）
|
||||
Local/Cluster 已增加同构 `GET /api/v3/projects/{projectId}/runs/{runId}/attempts/{attemptId}/log`、
|
||||
`run.log.read`/`artifact.read`。请求只接受 Project/Run/Attempt identity 与 offset/length，不接受 Artifact ID、路径、URI、bucket
|
||||
或 object key；Policy、durable audit 与 credential confirmation 先于 metadata 和存储访问，拒绝、不存在与跨 Project 对外统一
|
||||
遮蔽为 404。Runtime Core 复用现有 Run/Attempt authority；Local 使用私有文件 Range 与 canonical truncation fact，Cluster 在既有
|
||||
immutable S3 HEAD/checksum 后执行 ETag-fenced Range GET；缺少 COPY 后 `ChecksumSHA256` 的 S3-compatible provider 失败关闭。共享硬
|
||||
上限 256 KiB，但 Edge 默认 16/最大 32 KiB、Standalone 最大 32 KiB、Cluster 默认 64/最大 256 KiB，以满足 Local 64 KiB JSON
|
||||
response 和低配设备固定内存门。Runtime Core 494/494、Local Execution 35/35、Local API 45/45、Local Application 45 pass/4
|
||||
skip、Cluster Control 215 pass/2 skip；完整 18-package 门退出 0，backend 1,163 pass/2 skip/0 fail。workspace 保持 18 package/
|
||||
1,045 source、1,027 nested/18 reviewed root entry，无 single-source/shallow package，package/dependency/Edge import boundary 全绿。
|
||||
14 个 artifact 全部 compatible；Edge Application API 为 3,596,457 bytes/423 files/84 modules，仍不包含 Cluster、PostgreSQL 或
|
||||
AWS SDK。当前 arm64 Local image 为 10 package/385 files/3,331,225 bytes；Edge 128 MiB/64 PIDs 与 Standalone 256 MiB/256 PIDs
|
||||
均 graceful stop、20 events、SQLite integrity `ok`。固定摘要 MinIO KMS/SSE 条件 promotion、重放、HEAD/Range 集成通过；
|
||||
PostgreSQL 18.4 arm64 HA 通过 112 gates、timeline `1→2`，报告 SHA-256 为
|
||||
`61bea38e3a4f66884b9642c2fc1944dd7084f70ceb358d6d3fea8c11d5d33b65`。本批未新增 package、第三方依赖、migration、table/index、
|
||||
连接、listener、timer、watcher 或 cache；headless Local 与 MCP 不加载该能力，retention/tombstone 仍由 ADR-0027 后续闭环。
|
||||
- D-288/ADR-0376（已接受）
|
||||
Local/Cluster 固定同构 `POST /api/v3/projects/{projectId}/tasks/{taskId}/runs`、`task.start`/`run.start`，共享
|
||||
`qinglong/task-start@v1` 只接受 canonical mutation UUID、expected Task revision 与 contentDigest。Project/RoleBinding
|
||||
@@ -5574,7 +5590,7 @@ Dispatcher 只能把任务分配给满足 required 条件的 Worker。当没有
|
||||
- ADR-0120 已实现具体 S3-compatible shared Artifact store：body 先一次流式写入带 SHA-256/SSE 的共享临时 key，复验后用 destination `If-None-Match: *` server-side Copy 发布 opaque 永久 key；最终 HEAD 同时校验 S3 checksum、identity metadata、length 与 truncation。Put/Copy 丢响应和并发 winner 只由 inspect 裁决，临时 owner nonce 防止碰撞误删；未配置 SSE/KMS 不提供明文 fallback。该 adapter 已通过启用 static KMS 的真实 MinIO 验证，但不会因安装依赖而自动打开 Worker ingress 或 execution lifecycle。
|
||||
- ADR-0121 已实现默认关闭的 `@qinglong/worker-runtime/production`：它只在 `worker` Profile 下装配单 journal owner、单 mTLS Agent、完整 Offer-to-completion execution graph、Edge/Node 有界预算与一个 `unref` cadence，并在返回 active 前完成 startup reconciliation。shutdown 先 Abort Pull 但保留 owner，只有 Session 已 durable draining 且所有记录 settled 才释放资源；timeout、recovery 或 Session drain 失败均保留 authority 供重试。完整 Session heartbeat、证书/`ql3w` credential 与部署配置仍由外层产品生命周期补齐。
|
||||
- `next` 已按 ADR-0021 增加默认不可达的 Worker offer inbox：私有原子文件 journal 默认/硬上限为 64/1024 条、单条 192 KiB、分页最多 64 条；相同 offer 在进程内并发合并、跨重启按稳定 ID/digest 去重，过期 Lease、替换 Session、draining 新 claim 和冲突 payload 在 spawn 前拒绝。状态按 `accepted → starting_acknowledged → launching → started → running_acknowledged` 持久化；`launching` 是禁止自动二次 spawn 的 crash barrier，`started` 只重放 running ACK。文件 root 必须显式取得单 owner lease，默认 stale 30 秒且只有一个 `unref` 刷新 timer；双 owner、未持锁操作和 lock compromise 全部 fail closed。Inbox lifecycle 固定 acquire→audit→hold，审计失败或预算耗尽先释放再拒绝启动。启动审计默认扫描 4×16 条，硬上限 16×64，输出不含 token/命令，并把 `launching/recovery_required/started/running_acknowledged` 阻断为待恢复 ownership。Receiver 已在 spawn barrier 前校验 ephemeral completion callback，只把 sequence 与 token SHA-256 成对持久化，原 token 不进入 journal；旧记录继续可读但不能认证 receipt。其后的 recovery evidence pass 固定 receipt-first：SHA-256 authenticator 同时校验 sequence 并 constant-time 比较 digest，随后校验 Run/Attempt/start time，receipt 缺失才探测 durable process identity；进程 exited 后立即重读并可做最多 5 秒的有界 grace，invalid/mismatch/unsupported/I/O 均保守分类。结果不含 capability，不调用 start/stop/ACK/remove，并明确区分 current、替换 Session、offline、Worker/Run Lease 过期及控制面 terminal。该 adapter 不增加数据库连接、watcher 或每任务 sidecar。
|
||||
- 当前已具备认证 ExecutionSpec delivery、跨重启 delivery attempt/退避、真实 activation/Artifact/completion/lease-control HTTPS client、批量 Secret delivery、Secret-before-Artifact materializer、受审 POSIX Executor、流式本地 Artifact source、S3-compatible shared Artifact store、PostgreSQL completion/lease-control/cancellation/convergence repository、固定 Cluster Control 生产路由组合，以及在同一 package journal 上推进 `starting_acknowledged → launching → started → running_acknowledged`、从可信回执收敛 completion、按 durable deadline/失租停止精确进程的 Processor/协调器和具体 Worker production execution composition，不再要求把 accepted record 双写到旧 Receiver。具体 KMS/Vault Secret provider、Artifact range read/retention、对象存储 credential/lifecycle 产品配置，以及完整 Worker Session heartbeat、证书/`ql3w` credential 与 deploy bootstrap 仍未完成；execution subpath 因此继续默认关闭。HTTP 成功或本地 delivery receipt 仍不得冒充执行已经开始。
|
||||
- 当前已具备认证 ExecutionSpec delivery、跨重启 delivery attempt/退避、真实 activation/Artifact/completion/lease-control HTTPS client、批量 Secret delivery、Secret-before-Artifact materializer、受审 POSIX Executor、流式本地 Artifact source、S3-compatible shared Artifact store、Project-scoped Artifact range read、PostgreSQL completion/lease-control/cancellation/convergence repository、固定 Cluster Control 生产路由组合，以及在同一 package journal 上推进 `starting_acknowledged → launching → started → running_acknowledged`、从可信回执收敛 completion、按 durable deadline/失租停止精确进程的 Processor/协调器和具体 Worker production execution composition，不再要求把 accepted record 双写到旧 Receiver。具体 KMS/Vault Secret provider、Artifact retention/tombstone、对象存储 credential/lifecycle 产品配置，以及完整 Worker Session heartbeat、证书/`ql3w` credential 与 deploy bootstrap 仍未完成；execution subpath 因此继续默认关闭。HTTP 成功或本地 delivery receipt 仍不得冒充执行已经开始。
|
||||
|
||||
SQLite `Workers` adapter 只用于单控制面协议孵化、竞争测试和 standalone 未来连接 Worker 的兼容路径；它不是多副本 cluster-control 的存储方案。完整决策见 ADR-0012。
|
||||
|
||||
@@ -6285,7 +6301,7 @@ content-addressed bundle 与 entry 以 no-replace 发布，edge/standalone 每
|
||||
authenticated publisher trust provision、addition-only overlap rotation 与 generation
|
||||
恢复，并统一按不可变 lock creation time 校验签名 key lifetime；旧 key
|
||||
retirement/紧急 revoke、在线 OCI fetch、HTTP/UI、备份/rekey、Approval/retry、
|
||||
Artifact retention/read、外部
|
||||
Artifact retention/tombstone、外部
|
||||
deployment/cutover controller 仍未接入；因此可以宣称 base headless target executable
|
||||
及本机 queued package recovery consumer/publisher 已可达，但不能宣称 2.x cutover、
|
||||
在线插件市场或完整产品 surface 已完成。
|
||||
@@ -6296,6 +6312,8 @@ ADR-0068/0087 增量：`@qinglong/local-execution/recovery` 只依赖 runtime-co
|
||||
|
||||
ADR-0069/0070/0071/0072/0073/0074/0076/0077/0078/0079/0083/0084/0085/0086 增量：`@qinglong/local-process` 只依赖 runtime-core，拥有 receipt/identity 协议、digest-bound POSIX launcher、exact controller 与数据库索引 cleanup；local-execution 与 local-execution-control 单向依赖 local-process，local-run-recovery 再依赖统一控制包，local-dispatch 单向依赖 local-execution，local-secret 与 local-identity 生产均只依赖 runtime-core，短生命周期 local-secret-admin 只依赖 local-secret/runtime-core，local-owner-bootstrap 只依赖 local-identity/runtime-core，local-owner-console 再以专用 SQLite bootstrap 子入口组合 POSIX proof 与有界 staged secret outbox，且 authority 依赖均受审计隔离。launcher 在 spawn 前登记 exact Run/Attempt，身份捕获失败停止进程组，wrapper 清除 callback 环境并 no-overwrite 发布回执。独立 coordinator 已在 spawn 前原子提交 dispatching/starting 与 callback digest，spawn 后原子提交 handle/PID/running；ownership 写失败先复验身份并 TERM/KILL，stop 不确定时保留 starting。SQLite reviewed migration `0003/0004` 建立 journal，`0005/0006` 加入 dispatch plan并推进 capability v3，`0007/0008` 加入 encrypted Secret envelope并推进 v4，`0009/0010` 加入 ownerless Project/RoleBinding/security audit 和原子授权 mutation并推进 v5，`0011/0012` 加入 ownerless stable Identity/append-only API credential 并推进 v6，`0013/0014` 加入一次性 provisioning/digest-only challenge 并推进 v7；非法回执先持久化 quarantine intent，终态文件清理后才 resolve。application 只拥有一个 `unref` execution-control timer，同时驱动 completion notification、deadline/cancellation 单页扫描与低频 cleanup：edge 控制 5 秒×4、cleanup 5 分钟×8，standalone 控制 1 秒×32、cleanup 1 分钟×32；显式 dispatcher 每次只处理 edge 4/standalone 16 个候选并最多启动一个。Artifact allocator、Secret-first materializer、AES-256-GCM provider、数据库外显式 keyring preflight 与 4/64 MiB output hard quota已接入；稳定本机 credential authenticator、一次性 provisioning、原子首 Owner claim、POSIX 私有文件证明组合根以及 credential/challenge 的 prepare→commit→publish→recovery 协议已孵化，pepper/credential rotation/recovery 和两个 GC 核心已完成，`ql3-owner-gc` 与 transport-free `ql3-owner` 产品 CLI 已提供；但 fresh database/pepper setup、Secret/Project/Role/Approval 管理入口、备份/rekey、retry、Artifact retention/read、HTTP admission 与部署 controller 完成前仍保持默认生产不可达。
|
||||
|
||||
ADR-0377 已取代上段的 Artifact read/HTTP 缺口：Local/Cluster 的 Project-scoped、Profile-aware range read 已进入可选产品面；仍未闭环的是 retention/tombstone、管理 UI/CLI 与部署 controller。
|
||||
|
||||
ADR-0087 execution 合并增量：application 常驻闭包只安装 `@qinglong/local-execution` 的四个显式 subpath，不存在 package 根入口；内部依赖只允许 recovery→control 与 dispatch→execution。构建前 stale-dist 清理后最大 application 为 2,351,747 bytes、489 files、58 modules，仍低于 4 MiB/512 files/16 MiB 门禁，并较合并前减少 38,051 bytes 与 6 files。
|
||||
|
||||
### 11.5 LocalProcess completion receipt 与日志存活
|
||||
@@ -8432,7 +8450,7 @@ flowchart LR
|
||||
|
||||
> ADR-0058 至 ADR-0092 的以下段落是历史快照，其 PostgreSQL 数字与“下一切片”已由 ADR-0104/0105 及后续 Accepted ADR 取代；资源、Package 和物理证据边界仍保留作为演进记录：PostgreSQL 当时 baseline 为 11 条 reviewed migration、capability v10、19 张表和 migration/runtime/admin/worker-ingress 四角色；本机 SQLite 为二十八条 reviewed migration、capability v14、二十六张 owned table，并新增正式 TaskDefinition head/immutable revision Repository。当前有二十七个受审 3.0 importer：Owner maintenance 已合并为两个权限隔离 subpath，execution/control/recovery/dispatch 已合并为 `@qinglong/local-execution` 四个单向 subpath，bootstrap/credential-recovery 已合并为 `@qinglong/local-owner-ceremony` 两个互不依赖 subpath；三个 package 都不提供聚合根入口，hard cap 已同步降为 27。一次拓扑 build/test 会先清理已登记 QL3 package 的 stale dist；27 包全量测试、dependency/source boundary、联网 vulnerability audit 与六种 Profile 制品门禁均通过。ADR-0090 让 production packlist 只发布 JS、`.d.ts` 和受审 assets/drizzle，不再携带开发 map；ADR-0092 后当前最大 application 为 1,691,009 bytes、267 files、61 loaded modules，最大抽样 RSS delta 11,780,096 bytes，低于 4 MiB/512 files/16 MiB。所有 package build 已变为 self-only，全量从约 198 次编译降至精确 27 次，clean 状态单包测试仍按依赖闭包工作。资源门禁已拆为原生 Linux x64/arm64 的 128 MiB router stress、256 MiB Edge release guard 与 512 MiB Cluster control guard，并从容器内部验证 cgroup v2、零 swap/OOM、非 root、只读挂载、seccomp 和 `NoNewPrivs`；本轮本地原生 arm64 三档已通过，x64 等待远端 CI，所有档位均明确不是物理设备或生产容量承诺。物理 Edge candidate recorder 已绑定设备 manifest、实际 Linux/存储环境、Edge/SQLite 基准与 no-replace SHA-256 报告，并会拒绝容器/VM；同设备同 boot 的 idle sampler、不主动填盘的专用文件系统 fault probe，以及通过正式 Repository 写入/扫描 100/1000/10000 个 `qinglong/command@v1` TaskDefinition 的规模记录协议均已具备契约门禁，但尚未取得固定实机报告或 signature。ADR-0091 已冻结 1–32 个 exact descriptor 的不可变 TaskSpec semantic registry、内建 command v1 和本机写前门禁；历史 revision 在 provider 缺失时仍可读。ADR-0092 已实现绑定 source revision/content digest 的 Profile-neutral command plan 与确定性本机 context/execution 映射，且保持 subpath-only。
|
||||
|
||||
> 下表 PR-1 的旧“未完成”累计文字中，`completion/cancellation/timeout` 已由 ADR-0072 取代并闭环；当前本机剩余项是人工 recovery、retry 产品策略、Artifact retention/read、HTTP/CLI/UI 与部署 controller。远端 Worker completion 与 Workflow cancellation 的核心数据库链和 HA 重放已闭环；仍缺 production ingress/internal runtime port、expiry/retry lifecycle、部署启动装配与真实 Kubernetes 故障证据，不能与本机结论混用。
|
||||
> 下表 PR-1 的旧“未完成”累计文字中，`completion/cancellation/timeout` 已由 ADR-0072 取代并闭环，Artifact range read 已由 ADR-0377 闭环；当前本机剩余项是人工 recovery、retry 产品策略、Artifact retention/tombstone、HTTP/CLI/UI 与部署 controller。远端 Worker completion 与 Workflow cancellation 的核心数据库链和 HA 重放已闭环；仍缺 production ingress/internal runtime port、expiry/retry lifecycle、部署启动装配与真实 Kubernetes 故障证据，不能与本机结论混用。
|
||||
>
|
||||
> 下表 PR-0 的累积长文本仍含“四角色、21 条 migration、capability v20、21 项 HA”历史短语；当前权威基线应读取为六角色、23 条 migration、capability v22/36 表和 23 项 physical HA gate。ADR-0145/0146 又增加默认关闭的 manager-only TLS 1.3 management process、可选双副本 operation 与 durable distributed quota；它仍缺全副本重启 keyset anti-rollback 和真实 IdP/live ingress，所以“受认证管理入口”保持“已孵化但生产失败关闭”。
|
||||
|
||||
@@ -8468,12 +8486,12 @@ flowchart LR
|
||||
| 切片 | 当前状态 | 已具备 | 尚未接入/门禁 |
|
||||
| --- | --- | --- | --- |
|
||||
| PR-0 Migration/Cluster 基线 | In progress | SQLite/PostgreSQL reviewed migration、checksum/rollback/schema ownership/readiness 与 Node 20/24 × x64/arm64 CI 基线；`runtime-core` 提供共享领域 contract，`cluster-postgres` 提供 exact-pinned Pool、四角色、21 条 migration、capability v20/35 表、可写主库、受审 function/role activation check、完整 cluster repository 及显式 admin-only Plugin Package installation和 Approved Action authority，`cluster-admin` 提供短生命周期发布 authority、POSIX/Kubernetes Secret adapter、标准 OCI stage verifier、exact-registry credential provider、有界恢复与 caller-driven Package Approved Action dispatcher，`local-admin` 以相同 handler 组合 edge/standalone 单 SQLite authority；`cluster-control` 提供 readiness-first HTTP/mTLS admission、startup recovery、数据库时钟 row-lease scheduler 和 Remote Worker 控制面。Package admission 已在两个方言的同一事务中强绑定 immutable proposal、exact executing snapshot、Project Policy fence 与数据库观察时钟，并以 receipt 收敛 COMMIT response loss；dispatcher 固定 start 前 retry、start 后 durable inspection，edge/standalone/cluster 默认单批 1/4/16 且无 timer。PostgreSQL 18 physical-promotion 门禁已在本机 arm64 完成 streaming、`remote_apply`、受控复制链分区、fence-before-promote、timeline 1→2、新 generation 同步策略、新 slot、旧主 `pg_rewind`/只读 sync rejoin、端点延迟切换、双 control 摘流/激活，以及 scheduler、credential v1/v2/v3/v4、completion、cancellation 和通用事务不确定窗口的精确收敛。ADR-0126/0127 进一步完成独立 process/migration binary、三 workspace package/五 production root 的 builder/production 双 lock、实际非 root/只读 Docker smoke、双副本 QL3 Kustomize、migration/runtime authority 分离与有界私有 CA 文件绑定；本地 PostgreSQL 18 TLS 1.3 门禁已验证旧 CA、overlap、新 CA、SIGHUP 端点证书切换和逆序回滚，并接入 x64/arm64 CI；ADR-0128 已完成 control 43 外部+3 内部与 admin 85 外部+3 内部 component 的精确 CycloneDX、两种真实 arm64 镜像 inventory 对账、control 真实本地 amd64/arm64 OCI + 逐平台 SPDX/SLSA digest graph、双镜像原生 x64/arm64 image/OCI CI matrix、两个独立 GHCR repository 的 digest-bound Cosign/GitHub attestation 发布契约与发布后 manifest/signature/provenance/CycloneDX 四重自验证门；ADR-0129 已完成 CloudNativePG 1.30.0/PostgreSQL 18.4 digest lock、三实例同步 HA、四 DatabaseRole/Database owner、离散凭据、`-rw` endpoint、`ca.crt` 投影、Kustomize/audit 门及四节点 Kind worker-loss live contract/CI 定义；ADR-0137 至 ADR-0141 已完成 Plugin Package PostgreSQL parity、Approved Action 双方言 durable authority、immutable Package proposal、execution/start-barrier、Package 双方言原子 admission receipt、单用途 Project/Policy lock authority、跨 adapter 合同、标准 OCI resolver、exact-registry credential provider、ConfigMap CAS publisher、独立 admin image、最小权限 Job/RBAC 与真实 Kind 双 Pod resourceVersion/RBAC live gate，并由 21 项本机物理 HA 门复验 PostgreSQL authority。ADR-0230 把 Workflow frontier 与 Task Attempt admission 接入既有 production Scheduler cadence；ADR-0231 又在同一 production process 中以独立 mTLS listener/Worker Pool 和内部 runtime capability port 装入 Remote Worker 全套无 Secret transport，保持 20 包、单 Scheduler timer 与显式权限 subpath | PostgreSQL 18 × x64/arm64 远端 CI 成功记录；取得一次 CloudNativePG live job 成功证据并继续完成 CA overlap rotation/rollback、Barman/WAL archive restore、Kubernetes HA control-plane/failover、基础设施 STONITH、真实 Pod/raw-wire 分区、多 Pod 与 Session replacement 竞争；stale lock/temp 修复与 pepper retire/reference/GC；PostgreSQL Package admission 过期租约真库门、资源 generation consumer 和受认证管理入口；admin 真实双架构 OCI 记录、两个 image 的实际远端 GHCR manifest digest、Cosign/GitHub attestation 独立 verify 记录、漏洞/容量证明；Cluster Secret provider、Remote Worker expiry/retry lifecycle 与启动装配；管理 CLI/API/UI、审批与 rate limit；schedule 指标/告警；正式停写与双数据库 backup manifest；固定物理 edge 基线和完整 2.x API 契约样本 |
|
||||
| PR-1 Run Schema | Incubating | Run/RunAttempt/RunEvent schema、nullable cancel request 与 Attempt deadline 字段及恢复索引、CancellationDispatch 状态/version/lease/backoff schema、Repository port、临时 Sequelize adapter、统一事件大小/分页上限、跨 adapter RunRepository contract suite（原子事务、回滚、Run/Attempt/RetryPolicy CAS、唯一错误、分页与取消恢复）；ADR-0041 的 `pg-0003-run-retry-policy`、capability v2、driver-neutral PostgreSQL Run Repository 与真实 `pg.Pool` 上的共享 Repository/rollback/SQLSTATE contract；ADR-0063/0069/0071/0073/0074/0076 的独立 Node 24 local-sqlite typed schema、十二条 reviewed migration、capability v6、共享 operation authority、readiness/RunRepository/API credential repository/receipt journal/dispatch plan/encrypted Secret envelope/Project Policy/security audit/authorized mutation/stable Identity catalog、Drizzle↔真实 catalog table/column/index/CHECK/FK lockstep、base/adopted/application edge/standalone 产物门禁；ADR-0064 的 legacy baseline/plan digest、Online Backup recovery、side-by-side target migration、staged manifest、双库栅栏 activation、source 生命周期写栅栏、target stable identity 和重启语义；ADR-0065 的独立 cutover authority、外部副作用停机 evidence、append-only journal、start/restart/stop barrier 与 unknown→manual_required 收敛；ADR-0066 的 adopted storage→Run reconciliation→receipt maintenance→domain recovery→lifecycle→admission application gate、严格有界 recovery summary 与 admission-first reverse stop；ADR-0067 的 SQLite 事实驱动 Run 候选源、256 条硬上限、截断失败关闭和唯一 Repository authority；ADR-0068 的 receipt-first Reconciler、callback token/sequence fence、exact local-process identity、Attempt/Run/双 Event 原子终态推进和最终 verifier；ADR-0069 的 local-process 单向包边界、pre-spawn journal、受审 POSIX launcher、immutable receipt、exact identity 和 Profile-aware cleanup lifecycle；ADR-0070 的独立 local-execution、spawn 前后双 transaction CAS、callback digest、exact stop 补偿与 fail-closed starting 保留；ADR-0071 的独立 local-dispatch、不可变 revision/context、Secret-first materializer、Profile Artifact admission、4/64 MiB output hard quota 和窄 application facade；ADR-0073/0074 的 Project-bound SecretRef、AES-256-GCM、外置 keyring 生命周期、双 SQLite authority CAS、application preflight、强 Principal/Policy 和 envelope+audit 原子提交；ADR-0086 的本机 Owner provisioning/challenge/claim/delivery acknowledgement/credential recovery CLI | fresh database/pepper setup、credential rotation/GC 运维编排与 Secret/Project/Role/Approval 管理 CLI/API/UI、备份/rekey、2.x/target process controller、人工 recovery、target 写后 reconciliation 与完整 cutover/rollback 演练；retry 产品策略、Artifact retention/read stack、具体本机 lifecycle 和 target executable；Linux x64/arm64、PID namespace、断电与固定路由设备门禁；PostgreSQL 16/18 双连接并发与 failover integration；Task revision/context 跨方言 contract/并发压力与引用感知 retention、Keyv 数据迁移 |
|
||||
| PR-1 Run Schema | Incubating | Run/RunAttempt/RunEvent schema、nullable cancel request 与 Attempt deadline 字段及恢复索引、CancellationDispatch 状态/version/lease/backoff schema、Repository port、临时 Sequelize adapter、统一事件大小/分页上限、跨 adapter RunRepository contract suite（原子事务、回滚、Run/Attempt/RetryPolicy CAS、唯一错误、分页与取消恢复）；ADR-0041 的 `pg-0003-run-retry-policy`、capability v2、driver-neutral PostgreSQL Run Repository 与真实 `pg.Pool` 上的共享 Repository/rollback/SQLSTATE contract；ADR-0063/0069/0071/0073/0074/0076 的独立 Node 24 local-sqlite typed schema、十二条 reviewed migration、capability v6、共享 operation authority、readiness/RunRepository/API credential repository/receipt journal/dispatch plan/encrypted Secret envelope/Project Policy/security audit/authorized mutation/stable Identity catalog、Drizzle↔真实 catalog table/column/index/CHECK/FK lockstep、base/adopted/application edge/standalone 产物门禁；ADR-0064 的 legacy baseline/plan digest、Online Backup recovery、side-by-side target migration、staged manifest、双库栅栏 activation、source 生命周期写栅栏、target stable identity 和重启语义；ADR-0065 的独立 cutover authority、外部副作用停机 evidence、append-only journal、start/restart/stop barrier 与 unknown→manual_required 收敛；ADR-0066 的 adopted storage→Run reconciliation→receipt maintenance→domain recovery→lifecycle→admission application gate、严格有界 recovery summary 与 admission-first reverse stop；ADR-0067 的 SQLite 事实驱动 Run 候选源、256 条硬上限、截断失败关闭和唯一 Repository authority；ADR-0068 的 receipt-first Reconciler、callback token/sequence fence、exact local-process identity、Attempt/Run/双 Event 原子终态推进和最终 verifier；ADR-0069 的 local-process 单向包边界、pre-spawn journal、受审 POSIX launcher、immutable receipt、exact identity 和 Profile-aware cleanup lifecycle；ADR-0070 的独立 local-execution、spawn 前后双 transaction CAS、callback digest、exact stop 补偿与 fail-closed starting 保留；ADR-0071 的独立 local-dispatch、不可变 revision/context、Secret-first materializer、Profile Artifact admission、4/64 MiB output hard quota 和窄 application facade；ADR-0073/0074 的 Project-bound SecretRef、AES-256-GCM、外置 keyring 生命周期、双 SQLite authority CAS、application preflight、强 Principal/Policy 和 envelope+audit 原子提交；ADR-0086 的本机 Owner provisioning/challenge/claim/delivery acknowledgement/credential recovery CLI；ADR-0377 的 Local/Cluster 同构、Profile-aware、Project-scoped Artifact range read | fresh database/pepper setup、credential rotation/GC 运维编排与 Secret/Project/Role/Approval 管理 CLI/API/UI、备份/rekey、2.x/target process controller、人工 recovery、target 写后 reconciliation 与完整 cutover/rollback 演练；retry 产品策略、Artifact retention/tombstone stack、具体本机 lifecycle 和 target executable；Linux x64/arm64、PID namespace、断电与固定路由设备门禁；PostgreSQL 16/18 双连接并发与 failover integration；Task revision/context 跨方言 contract/并发压力与引用感知 retention、Keyv 数据迁移 |
|
||||
| PR-2 Run 状态机 | Incubating | 纯转换表、终态/时间/错误/执行器元数据规则、Run version 与 event sequence CAS、事务性 RunCommandService、回滚测试 | 重复 Worker callback/fencing、并发数据库压力测试、Primary 执行链接入 |
|
||||
| PR-3 Executor 端口 | Incubating | ADR-0003、ExecutionSpec/Context/Handle/Result、Executor port、LocalProcessExecutor、进程组取消/超时升级、流式背压、Legacy Cron spec builder、真实进程 contract tests、可复现 edge 基准入口 | 固定 edge/多架构设备基线、Legacy builder 与 makeCommand 差异审计、Primary 生产流量接入 |
|
||||
| PR-4 Shadow Run | Incubating | origin 三态策略；默认关闭的 `QL3_SHADOW_ORIGINS`；manual 与 scheduled_node 现有 ChildProcess 旁路观察；每个 worker 懒加载；Run/Attempt/Event 影子生命周期；稳定 task revision 与有界日志引用；同 worker 有界注册表和跨 worker 持久化候选关联；stop all/stop instance、Shell callback、乱序/迟到/歧义处理；失败开放和契约测试 | `ScheduleService.runTask` 覆盖的 scheduled_system/subscription/system/script 等高风险入口；启动后 Reconciler、差异报表、可采集指标、资源压力、回滚演练和 Primary 门禁 |
|
||||
| PR-5 Primary LocalExecutor | Incubating（默认不激活，仅 manifest-gated manual） | runtime-owned Run 创建器；持久化先于 spawn；Run/Attempt 完整成功、失败、取消、超时与 lost 闭环；Executor handle 身份校验；spawn 后激活写失败的 stop+lost 补偿；completion rejection 安全收敛；独立 Primary 幂等查询与唯一索引竞态裁决；durable `run.cancel_requested`、stop-before-signal、首次请求幂等、晚到完成裁决与待取消有界恢复查询；最多 64 条一页的 cross-worker cancellation source；独立 CancellationDispatch Repository 原子 claim/result、lease expiry 接管、owner/token/version fencing、指数退避与结果 RunEvent；最多 64 页的单周期 cancel supervisor；显式 start/stop、无重叠、错误隔离、停止等待有上限且 timer unref 的 lifecycle runner；Linux durable handle 的 PID/boot/start ticks/process-group 复验与 TERM/KILL controller；完整有界分页且 fail-closed 的 startup Reconcile supervisor；RunningInstance nullable `run_id/attempt_id` 关联；Primary 专用组合 Repository 在同一 SQLite 事务提交前投影 Crontab/RunningInstance，失败整体回滚；有界且防穿越的 legacy log output ref；manual owner seam、真实本机装配、单 spawn/fail-closed；严格 manual-only rollout manifest loader、短期审批/gate、配置哈希审计；HTTP worker 已接轻量 lazy bootstrap，accepted 后按 receipt-first reconcile→completion receipt lifecycle→timeout intent lifecycle→cancel dispatch lifecycle→router 顺序激活，失败撤销，监听失败和 shutdown 有界停止；Primary timeout 在 spawn 前持久化绝对 deadline，有界 source/requester/supervisor 只提交 timeout 意图并复用 CancellationDispatch；代码级 edge/standalone Profile 为各 lifecycle 提供不同 cadence 与页上限，cluster-control/worker 拒绝误装本机 SQLite Primary；统一 CompletionService 原子提交 Attempt/Run/双 Event，spawn 前保存 callback token hash、终态推进 sequence，实时回调与 receipt consumer 共享入口并覆盖两个清理 crash window；manual Primary 已接入受限 POSIX launcher、`0600` direct-file stdout/stderr、父进程退出后续写、不可覆盖 receipt 生产、回执环境清除、TERM 转发等待及 live transaction 后清理；Startup Reconciler receipt-first 双检查并在确定 exited 后执行 profile 化的单次 50/100 ms publish grace；`0007` 独立 CompletionReceiptJournal 在 spawn 前登记、为升级前 active Attempt 补登记并驱动周期扫描，使终态残留继续可发现；确定无效的已知 Attempt receipt 先持久化隔离状态，再进入确定性私有分片 quarantine；终态 missing 与 quarantine 按 edge/standalone retention 有界清理；非 Journal 文件具备只读优先、固定分片/条目上限、overflow fail-closed、显式同盘隔离的 Node 24 运维 CLI；扫描具备页上限、resume cursor、timer unref、无重叠、有界 stop 和低敏计数；ENOSPC 与 launcher receipt 存储失败有代码门禁 | 部署配置写入/审批入口与用户可见状态；PostgreSQL CancellationDispatch adapter；cluster-control 生产启动拓扑；固定 edge/Linux 多架构与真实磁盘压力基线、完整 2.x API 契约和回滚演练 |
|
||||
| PR-7 Worker Session、Run Lease 与启动协议基础 | Incubating（默认关闭，独立入口显式 opt-in） | ADR-0012/0013/0014/0021/0057–0061/0108–0121/0231–0239；有界 capability/Placement/Dispatcher；SQLite 协议孵化与 PostgreSQL v9 Session/Run Lease/credential/attestation authority；immutable revision Placement、数据库时钟 keyset candidate、认证 Worker Pull、digest-only offer recovery；versioned capability-free ExecutionSpec response、stable claim 跨重启退避、单 owner 原子 inbox 准入与 TLS 1.3 mTLS/`ql3w` HTTPS client；同一 package journal 上 revision-fenced starting/spawn/started/running/completion 状态、callback digest、tagged no-spawn 与 ambiguous recovery；PostgreSQL starting/running/start-failure/completion 数据库权威事务、精确重放与 cancellation/timeout 优先终态；batch Secret delivery 在 Attempt advisory lock 下复验 Session/Lease/revision 完整围栏并复用单 Agent，Secret-before-Artifact materializer 将同一 log ID 交给 Executor/journal/running ACK；offer-scoped `wlog-*` 私有文件 spool、Edge/Node 容量策略、append/quota/path 防护、barrier 后 output ownership、受审 POSIX Executor、truncation fact、固定内存流式 source、认证 Artifact stream、共享 immutable store port、S3-compatible SSE/checksum/条件 promotion adapter 与 upload-before-completion 协调；用户取消 run.stop mutation 以数据库时间写 intent/Event 并在事务内复验 Project/RoleBinding fence；非执行取消 convergence lifecycle、运行期 expiry 与安全 lost retry 已接入 cluster-control 单一全局 cadence；完整 generation/version/token/Attempt fencing；独立最小权限 Worker ingress、CA/CRL 与连接 generation 热重载；offer journal、spawn barrier、receipt-first recovery；独立 `@qinglong/worker-runtime` 的本地 P-256 CSR、key/chain/trust 验证、generation + active pointer 安装和持久退避；默认关闭的 production process 已装配具体 execution graph、完整 Session heartbeat/drain/offline、direct-file bootstrap、单 Agent/单 cadence、startup reconciliation、证书 maintenance、transport fail-close/recovery 与 Edge/Node 有界预算；真实 PostgreSQL 18 + Linux Node 合约已覆盖 Run completion、credential 和 CA 双轮换且保持同一 Session；真实 K3s 合约已覆盖 TLS/credential Secret 分权、双对象 CAS、Recreate 顺序、identity generation 与单节点 PVC recovery；所有能力默认不可达且受 edge/cluster import audit 约束 | 具体 cert-manager/Vault/SPIFFE/离线 CA adapter 与模板、ingress reload controller、生产 RBAC、证书到期告警和 `ql3w` credential recovery 产品面；具体 KMS/Vault Secret provider、对象存储 credential/temporary lifecycle、range read/retention；Worker 管理 API；真实 Kubernetes 多节点 CSI/node-loss/production 360 秒 drain 与固定 edge 文件系统 suspend/时钟/断电、x64/arm64 资源门禁 |
|
||||
| PR-7 Worker Session、Run Lease 与启动协议基础 | Incubating（默认关闭，独立入口显式 opt-in） | ADR-0012/0013/0014/0021/0057–0061/0108–0121/0231–0239/0377；有界 capability/Placement/Dispatcher；SQLite 协议孵化与 PostgreSQL v9 Session/Run Lease/credential/attestation authority；immutable revision Placement、数据库时钟 keyset candidate、认证 Worker Pull、digest-only offer recovery；versioned capability-free ExecutionSpec response、stable claim 跨重启退避、单 owner 原子 inbox 准入与 TLS 1.3 mTLS/`ql3w` HTTPS client；同一 package journal 上 revision-fenced starting/spawn/started/running/completion 状态、callback digest、tagged no-spawn 与 ambiguous recovery；PostgreSQL starting/running/start-failure/completion 数据库权威事务、精确重放与 cancellation/timeout 优先终态；batch Secret delivery 在 Attempt advisory lock 下复验 Session/Lease/revision 完整围栏并复用单 Agent，Secret-before-Artifact materializer 将同一 log ID 交给 Executor/journal/running ACK；offer-scoped `wlog-*` 私有文件 spool、Edge/Node 容量策略、append/quota/path 防护、barrier 后 output ownership、受审 POSIX Executor、truncation fact、固定内存流式 source、认证 Artifact stream、共享 immutable store port、S3-compatible SSE/checksum/条件 promotion adapter、upload-before-completion 协调，以及 Local/Cluster 同构、Profile-aware、ETag-fenced range read；用户取消 run.stop mutation 以数据库时间写 intent/Event 并在事务内复验 Project/RoleBinding fence；非执行取消 convergence lifecycle、运行期 expiry 与安全 lost retry 已接入 cluster-control 单一全局 cadence；完整 generation/version/token/Attempt fencing；独立最小权限 Worker ingress、CA/CRL 与连接 generation 热重载；offer journal、spawn barrier、receipt-first recovery；独立 `@qinglong/worker-runtime` 的本地 P-256 CSR、key/chain/trust 验证、generation + active pointer 安装和持久退避；默认关闭的 production process 已装配具体 execution graph、完整 Session heartbeat/drain/offline、direct-file bootstrap、单 Agent/单 cadence、startup reconciliation、证书 maintenance、transport fail-close/recovery 与 Edge/Node 有界预算；真实 PostgreSQL 18 + Linux Node 合约已覆盖 Run completion、credential 和 CA 双轮换且保持同一 Session；真实 K3s 合约已覆盖 TLS/credential Secret 分权、双对象 CAS、Recreate 顺序、identity generation 与单节点 PVC recovery；所有能力默认不可达且受 edge/cluster import audit 约束 | 具体 cert-manager/Vault/SPIFFE/离线 CA adapter 与模板、ingress reload controller、生产 RBAC、证书到期告警和 `ql3w` credential recovery 产品面；具体 KMS/Vault Secret provider、对象存储 credential/temporary lifecycle 与 retention/tombstone；Worker 管理 API；真实 Kubernetes 多节点 CSI/node-loss/production 360 秒 drain 与固定 edge 文件系统 suspend/时钟/断电、x64/arm64 资源门禁 |
|
||||
| PR-8 Project/Policy/Approval Core | Incubating（默认拒绝、无生产业务执行入口） | ADR-0028；统一六类 ActorRef 与 exact-shape 校验；`0017` ownerless default Project 和 append-only versioned RoleBinding；owner/admin/operator/viewer 固定矩阵；Project 内 mutation 幂等、expected-version CAS、双 SQLite 连接竞争门禁；archived read-only、revocation、存储损坏 fail-closed；Agent 写/Secret/Tool `require_approval`；ADR-0047 把六类 subject、role/permission matrix 与 fence 抽到 runtime-core，`pg-0004-project-policy`/capability v3 建立 ownerless PostgreSQL baseline、严格 role/state CHECK、append-only runtime 权限、SERIALIZABLE Project lock、mutation replay、双连接单 winner 和 cluster admission authorizer；ADR-0049/`pg-0005` capability v4 建立 stable IdentitySubject、append-only digest-only API credential、真实 cluster bearer authenticator、write-only durable security audit 与最小权限 runtime role，且已验证 HTTP→credential→Policy→audit→handler 纵向链路；ADR-0051 建立 `/api/v3` 认证前 peer/global 双预算、transport-peer-only、无 timer 且有界内存的 overload shield；ADR-0027 Artifact authorizer adapter；ADR-0029 `AuthenticatedPrincipal` contract、`0018` digest-only versioned challenge、CSPRNG/TTL、同事务消费 challenge + 写首 owner、精确重放与双连接竞争/崩溃回滚门禁；ADR-0030 `0019` stable identity/binding、legacy HS384 + current-session membership、logout/platform/revoke/disable、single-factor 与损坏 fail-closed 门禁；ADR-0031 `0020` digest-bound ApprovalRequest、User-only decision、Project/Role version fence、精确 expiry/重放/并发裁决及同事务 immutable dispatch；ADR-0032 `0021` execution backfill、三表原子 consume、稳定 due keyset、claim/renew/start/result fencing、pre-start takeover/post-start recovery-required、attempt budget、handler inspect/digest barrier 和 bounded dispatcher；ADR-0033/`0022` control/resolution backfill、start/renew/completion 原子联动、稳定 recovery keyset、双 resolver claim/takeover、finding/result 精确重放、自动/人工终结、迟到 completion 单 winner 和 evidence-only bounded reconciler；ADR-0034/`0023` 首个 `run.create` canonical plan、Run/Attempt/Event/receipt 同事务、幂等 collision fail-closed、renew/终态 fence、真实 SQLite handler 与 automatic evidence provider；ADR-0035/`0024` 独立 `approval.recover` 矩阵、稳定 User + 五分钟强认证、Project/RoleBinding fence、human resolution + authorization fact 原子提交、撤权竞态与回滚门禁；ADR-0036 recovery-first 单 timer lifecycle、edge/standalone 独立 cadence/页预算、跨周期 cursor、非重叠与有界 stop；ADR-0074 以新的 Node 24 SQLite v5 ownerless Project/RoleBinding/audit authority 和独立 local-secret-admin 提供强 Principal、`secret.manage`、撤权 fence、envelope+allowed audit 原子提交及不回显语义；ADR-0086 以可信 POSIX console 和 staged delivery 完成本机首 Owner 产品 ceremony | fresh database/pepper setup 与安全迁移向导；`shareStore`/Express 到 authentication core 的 production migration；credential rotation/revocation API、mTLS/Worker enrollment、恢复码；Project/Role/Approval/Secret 管理 CLI/API/UI、audit retention/query/export/alert、preview Artifact/digest/immutable plan builder、真实 MFA/hardware adapter、人工 recovery API/UI/独立 rate limit 与审计事件、handler/provider registry、lifecycle startup/shutdown/指标/admission gate；PostgreSQL action/receipt/provider/recovery-authorization 与 OPA adapter、缓存 version 失效；Tool/Package/Secret/Shell 各自的 handler/evidence contract；Secret/Run/Tool/Workflow waiting_approval 全入口装配；完整回滚演练 |
|
||||
|
||||
PR-8 的本机最新增量由 ADR-0075/0076/0077/0078/0079/0080/0081/0082/0083/0084/0085/0086/0087 约束：Node 24 SQLite capability v13 已有 ownerless stable Identity、append-only digest-only API credential、credential-version pepper provenance binding、versioned pepper catalog/append-only activation generation、credential recovery/material GC ledger、delivery acknowledgement tombstone GC、共享 operation authority、Runtime exact-key authenticator、一次性 provisioning、digest-only challenge 与无 secret delivery acknowledgement ledger；共享 `ApiCredentialRecord` 与 PostgreSQL capability v10 也具备同名 key provenance。独立 `@qinglong/local-owner-keyring` 提供最多 8 key 的 no-replace provision、低敏 inspect、独立 backup、absence-only restore、exact-ID load 和仅供 GC 的 destructive 子入口；认证只接受 record 指定且 catalog 状态为 active/retired、摘要一致的 material，staged/recovery-required/缺失均 fail closed。独立 `@qinglong/local-owner-credential-recovery` 以不同 ID 签发新 credential，复用 staged outbox；只有 exact delivery digest 已持久化确认，才允许在 SQLite 事务中 append revoked 旧 version、pepper binding 与 audit。短生命周期 `@qinglong/local-owner-maintenance/pepper-gc` 负责双材料 crash-safe 销毁；`/acknowledgement-gc` 以内建 clock、文件 bridge-clear proof、source 终态与 30 天 replay/audit 保留裁决，在单事务写 audit/tombstone 并删除完整行，bootstrap 随后从 immutable source 验证 semantic digest 后重建 exact replay，credential/challenge 均不重新请求 entropy。ADR-0085/0086 抽出共享 `@qinglong/local-command-file` 叶子协议；`ql3-owner-gc` 开放两个 GC ceremony，`ql3-owner` 开放 provisioning/challenge/transport-free claim/delivery acknowledgement/credential recovery，均只输出低敏 JSON。专用 `@qinglong/local-sqlite/bootstrap|pepper-gc|acknowledgement-gc`、`@qinglong/local-owner-bootstrap` 与短生命周期 console/maintenance authority 仍保持默认不可达，产品 CLI 只能经 console facade 到达 Owner ceremony。fresh database/pepper setup、Secret/Project/Role/Approval 管理产品入口、Linux/容器/断电/真实 ENOSPC 实机证据和 legacy adapter 未完成；两个本机 `bin` 都不是可远程使用的 cluster 管理入口。
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
# ADR-0377：Profile-aware Run Attempt 日志范围读取
|
||||
|
||||
- 状态：Accepted
|
||||
- 日期：2026-08-12
|
||||
- 关联 RFC：QL-RFC-0001 D-289
|
||||
- 前置决策：ADR-0027、ADR-0367、ADR-0376
|
||||
|
||||
## 上下文
|
||||
|
||||
Task Start 已能在 Local 与 Cluster 创建可执行 Run，产品入口也能读取 Run、Event、Step 并请求取消，但调用方仍无法从同一 Project-scoped 产品面读取某个 Attempt 的执行日志。早期 `back/runtime` 已验证本机 metadata-first 授权、私有文件 Range 与截断三态，但它不在 QingLong 3.0 package 生产图中，也没有 Cluster 对象存储实现。
|
||||
|
||||
直接恢复 2.x 的 `path + file` 会把路径当作外部 capability；把整份日志编码进 JSON 会让内存与日志大小线性增长；把 Local 与 Cluster 强行放到同一存储 adapter，又会使控制面依赖 Worker 本地路径。低配路由设备和集群节点需要共享语义，但必须拥有不同的单次窗口与存储实现。
|
||||
|
||||
## 决策
|
||||
|
||||
### 1. 同一产品路由与只读权限
|
||||
|
||||
Local 与 Cluster 提供同构入口：
|
||||
|
||||
`GET /api/v3/projects/{projectId}/runs/{runId}/attempts/{attemptId}/log?offset={n}&length={n}`
|
||||
|
||||
- operation：`run.log.read`
|
||||
- permission：`artifact.read`
|
||||
- response schema：`qinglong/run-attempt-log-read-result@v1`
|
||||
- `offset` 默认为 0；`length` 默认为当前 Profile 窗口
|
||||
- 请求只接受 Project、Run、Attempt identity 和有界范围，不接受 Artifact ID、路径、URI、bucket 或 object key
|
||||
|
||||
Policy、durable security audit 与 credential confirmation 必须先于 Run/Attempt metadata 和 Artifact 存储访问。对非可信 HTTP 调用方，Policy deny、require-approval、跨 Project、不属于该 Run、legacy owner、错误 executor 和不存在均返回相同的 404，不形成存在性 oracle；真实结果只进入低敏审计。
|
||||
|
||||
### 2. 共享 application contract，不共享存储 adapter
|
||||
|
||||
`runtime-core` 定义唯一的 Run Attempt 日志读取服务和 byte-range port。服务复用现有 `RunRepositoryReader.findRunById/findAttemptById`，依次验证：
|
||||
|
||||
1. Run 属于请求 Project 且 `executionOwner=runtime`；
|
||||
2. Attempt 属于请求 Run；
|
||||
3. executor 与当前 Profile 一致；
|
||||
4. Attempt 绑定 canonical `local-*` 或 `wlog-*` Artifact ID；
|
||||
5. range 规范化且不超过调用 Profile 的上限；
|
||||
6. 只有上述数据库事实成立后才调用存储 reader。
|
||||
|
||||
Local adapter 从 opaque `local-*` ID 派生私有 shard，拒绝 symlink、非普通文件、非当前进程 owner、多 hard link、异常短读和 identity 不一致的 truncation fact。Cluster adapter 先用现有 immutable S3 HEAD 校验 identity digest、length、content type、checksum metadata，再执行单次有界 Range GET；控制面从不拼接 Worker 路径。
|
||||
|
||||
S3-compatible provider 必须在最终对象 COPY 后，通过启用 checksum 的 HEAD 返回 canonical `ChecksumSHA256`；只返回调用方可写 metadata 而不能独立证明对象内容的实现不在兼容集合内，adapter 必须失败关闭，不能退化为信任 metadata 或 ETag。
|
||||
|
||||
### 3. Profile 窗口和资源上限
|
||||
|
||||
共享协议硬上限为 256 KiB，但部署档位收紧实际窗口：
|
||||
|
||||
- Local Edge：默认 16 KiB，最大 32 KiB；
|
||||
- Local Standalone：默认/最大 32 KiB；
|
||||
- Cluster Control：默认 64 KiB，最大 256 KiB。
|
||||
|
||||
当前 HTTP transport 只返回 JSON，因此 content 使用 Base64，并同时返回 byte range、总长度和下一 offset。Local 的 32 KiB 上限保证编码和元数据仍低于既有 64 KiB response hard limit；Cluster 保持低于既有 1 MiB 默认 response limit。reader 只分配请求窗口，不启动 watcher、tail timer、cache、额外 listener、Pool 或 S3 client。
|
||||
|
||||
### 4. 状态与失败语义
|
||||
|
||||
- `200 available`：返回当前文件/对象快照内的有界 bytes、`start`、`endExclusive`、`totalBytes` 和可选 `nextOffset`；
|
||||
- `202 pending`：合法远端 Attempt 已绑定日志，但对象尚未在 upload-before-completion 协议中发布，或 Attempt 尚未产生 Artifact identity；
|
||||
- `404 artifact_not_found`：不存在、越权或 identity/executor 不匹配的统一外部遮蔽；
|
||||
- `503 artifact_unavailable`：数据库、文件安全检查、对象存储或证据一致性无法证明。
|
||||
|
||||
`truncated` 保持 `true | false | unknown`。Local 只相信 launcher canonical fact；Cluster 只相信 immutable S3 metadata；禁止从 size、尾部文本或退出状态推断。offset 等于或超过当前快照末尾时返回空的 available range，并把 start 收敛到 totalBytes，便于轮询而不制造 416 竞态。
|
||||
|
||||
### 5. 本批明确不闭环 retention
|
||||
|
||||
本批不新增 tombstone table、delete marker、retention lifecycle 或对象删除权限。ADR-0027 的 retained/missing 竞态和 ADR-0026 retention 仍是独立后续切片。没有 tombstone 时，Local 的已绑定文件缺失和终态 Cluster 对象缺失都按 503 处理，不能伪装成已执行 retention；因此 ADR-0027 在 retention 完成前保持 Proposed。
|
||||
|
||||
### 6. package 与部署边界
|
||||
|
||||
- 不新增 workspace package、第三方依赖、migration、table/index、数据库连接或常驻资源；
|
||||
- contract 放入 `runtime-core` 既有 Run 能力目录；Local reader 放入 `local-execution` 既有 Artifact 能力目录；S3 reader 扩展 `cluster-control` 既有 immutable store；
|
||||
- 默认 headless Local 不加载文件 reader；只有可选 Local API product surface 通过 application authority 获得已构造 capability；
|
||||
- Cluster 复用 production Worker artifact binding 的同一 S3 client/store；未配置对象存储时 route 保持存在并失败关闭为 503；
|
||||
- MCP 本批不读取日志，避免在 Agent 面扩大敏感数据能力。
|
||||
|
||||
## 被否决的替代方案
|
||||
|
||||
1. **恢复 `path + file`**：路径成为跨 Project capability，拒绝。
|
||||
2. **整文件返回或服务端 tail 长连接**：内存、连接和后台资源无界，不适合 Edge，拒绝。
|
||||
3. **固定所有 Profile 为 256 KiB**：Base64 后突破 Local 64 KiB transport 门，拒绝。
|
||||
4. **Cluster 读取 Worker 共享挂载**：绕过 immutable upload、checksum 与 fencing，拒绝。
|
||||
5. **为日志读取新建微包或 metadata 表**：既有 Run/Attempt authority 足够，本批不制造薄 package，拒绝。
|
||||
6. **将 missing 当 retention**：没有 durable tombstone 无法证明，拒绝。
|
||||
|
||||
## 验收
|
||||
|
||||
1. shared contract 覆盖 identity/range、Project/Run/Attempt/executor 绑定、pending/available/missing 与截断三态；
|
||||
2. Local adapter 覆盖 symlink、owner/link count、append snapshot、短读、空/越尾 range 和 fact identity 冲突；
|
||||
3. S3 adapter 覆盖 HEAD identity/checksum、精确 Range、abort、404、响应长度/Content-Range 漂移和终态缺失；
|
||||
4. Local/Cluster HTTP 覆盖 route/query、`artifact.read`、durable audit、deny/not-found 遮蔽、202/200/400/404/503；
|
||||
5. 默认 headless Edge closure 不增加；Local API 增量仍满足 64 KiB response、4 in-flight 与 128 MiB/64 PID 门；
|
||||
6. 完整 package/backend、dependency/package boundary、Profile artifact、真实 SQLite+HTTP、S3-compatible integration、PostgreSQL HA 与 Local image 门全部通过后，状态才可改为 Accepted。
|
||||
|
||||
## Accepted 证据
|
||||
|
||||
- Runtime Core 494/494、Local Execution 35/35、Local API 45/45、Local Application 45 pass/4 platform skip、Cluster Control 215 pass/2 conditional skip；完整 18-package build/test 退出 0。
|
||||
- backend 1,163 pass/2 skip/0 fail；package boundary、dependency boundary 与 Edge import audit 全部 compatible。workspace 仍为 18 package、1,045 source、1,027 nested/18 reviewed root entry，无 single-source 或 shallow package。
|
||||
- 真实 SQLite+HTTP 已覆盖认证、Policy、durable audit、credential confirmation、Attempt metadata 与本地私有文件 Range 的完整顺序；Local 文件 adapter 的 symlink、权限、hard-link、短读、append snapshot、越尾空读与 truncation identity drift 全部通过。
|
||||
- S3 单元门覆盖 checksum/metadata/ETag/Content-Range 漂移、短体与超长体；固定摘要 `minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e` 的真实 KMS/SSE、条件 promotion、重放、HEAD 与 Range GET 通过。不能在 COPY 后返回 `ChecksumSHA256` 的旧 provider 被实测失败关闭。
|
||||
- PostgreSQL 18.4 arm64 HA 通过 112 gates，timeline `1→2`，报告 SHA-256 为 `61bea38e3a4f66884b9642c2fc1944dd7084f70ceb358d6d3fea8c11d5d33b65`。
|
||||
- 14 个 Edge/Standalone Profile artifact 全部 compatible。Edge Application API 为 3,596,457 bytes/423 files/84 loaded modules，距 6 MiB 门仍有 2,694,999 bytes；依赖闭包不包含 Cluster、PostgreSQL 或 AWS SDK。
|
||||
- 当前源码构建的 arm64 Local image 为 10 package/385 files/3,331,225 bytes。Edge 128 MiB/64 PIDs 与 Standalone 256 MiB/256 PIDs 均在 non-root、read-only root、network-none 下 active、20 events、graceful stop，SQLite integrity 为 `ok`。
|
||||
- 本批没有新增 package、第三方依赖、migration、table/index、连接、listener、timer、watcher 或 cache；Artifact retention/tombstone 仍由 ADR-0027 后续闭环。
|
||||
@@ -24,6 +24,7 @@ import { createClusterControlRunReadRoute } from '../run/runReadRoute';
|
||||
import { createClusterControlRunListRoute } from '../run/runListRoute';
|
||||
import { createClusterControlRunEventListRoute } from '../run/runEventListRoute';
|
||||
import { createClusterControlRunStepListRoute } from '../run/runStepListRoute';
|
||||
import { createClusterControlRunAttemptLogReadRoute } from '../run/runAttemptLogReadRoute';
|
||||
import { createClusterControlTaskListRoute } from '../task/taskListRoute';
|
||||
import { createClusterControlTaskReadRoute } from '../task/taskReadRoute';
|
||||
import { createClusterControlTaskStartRoute } from '../task/taskStartRoute';
|
||||
@@ -69,6 +70,7 @@ export const PRODUCTION_CLUSTER_CONTROL_ROUTE_OPERATIONS = Object.freeze([
|
||||
'run.list',
|
||||
'run.events.list',
|
||||
'run.steps.list',
|
||||
'run.log.read',
|
||||
'run.cancel',
|
||||
'workflow.read',
|
||||
'workflow.run.read',
|
||||
@@ -195,6 +197,10 @@ export function createProductionClusterControlApplicationStack(
|
||||
input.runs,
|
||||
input.trustedToolStorage.stepRuns,
|
||||
),
|
||||
createClusterControlRunAttemptLogReadRoute(
|
||||
input.runs,
|
||||
input.workerRuntime?.runAttemptLogRead,
|
||||
),
|
||||
createClusterControlRunCancellationRoute(
|
||||
input.runCancellation,
|
||||
createEventId,
|
||||
|
||||
@@ -6,12 +6,19 @@ import {
|
||||
ChecksumMode,
|
||||
CopyObjectCommand,
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadObjectCommand,
|
||||
MetadataDirective,
|
||||
PutObjectCommand,
|
||||
S3Client,
|
||||
ServerSideEncryption,
|
||||
} from '@aws-sdk/client-s3';
|
||||
import {
|
||||
normalizeRunAttemptLogReadRange,
|
||||
type RunAttemptLogRangeReadResult,
|
||||
type RunAttemptLogReadIdentity,
|
||||
type RunAttemptLogReadRange,
|
||||
} from '@qinglong/runtime-core/run-attempt-log-read';
|
||||
import {
|
||||
MAX_REMOTE_WORKER_ARTIFACT_BYTES,
|
||||
REMOTE_WORKER_ARTIFACT_CONTENT_TYPE,
|
||||
@@ -31,7 +38,8 @@ const TEMPORARY_METADATA_SCHEMA =
|
||||
const SHA256_PATTERN = /^[a-f0-9]{64}$/;
|
||||
const BUCKET_PATTERN = /^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$/;
|
||||
const PREFIX_PATTERN = /^[A-Za-z0-9][A-Za-z0-9/_=-]{0,254}$/;
|
||||
const TEMPORARY_ID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const TEMPORARY_ID_PATTERN =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
|
||||
type S3SendClient = Pick<S3Client, 'send'>;
|
||||
|
||||
@@ -70,8 +78,7 @@ export function createS3ClusterRemoteWorkerArtifactClient(
|
||||
typeof options !== 'object' ||
|
||||
Array.isArray(options) ||
|
||||
!/^[a-z0-9][a-z0-9-]{0,62}$/.test(options.region) ||
|
||||
(options.endpoint !== undefined &&
|
||||
typeof options.endpoint !== 'string') ||
|
||||
(options.endpoint !== undefined && typeof options.endpoint !== 'string') ||
|
||||
(options.forcePathStyle !== undefined &&
|
||||
typeof options.forcePathStyle !== 'boolean')
|
||||
) {
|
||||
@@ -79,9 +86,7 @@ export function createS3ClusterRemoteWorkerArtifactClient(
|
||||
}
|
||||
return new S3Client({
|
||||
region: options.region,
|
||||
...(options.endpoint === undefined
|
||||
? {}
|
||||
: { endpoint: options.endpoint }),
|
||||
...(options.endpoint === undefined ? {} : { endpoint: options.endpoint }),
|
||||
forcePathStyle: options.forcePathStyle ?? false,
|
||||
});
|
||||
}
|
||||
@@ -116,6 +121,11 @@ type ArtifactAuthority = Readonly<{
|
||||
logArtifactId: string;
|
||||
}>;
|
||||
|
||||
type StoredArtifactHead = Readonly<{
|
||||
receipt: Readonly<RemoteWorkerArtifactReceipt>;
|
||||
eTag?: string;
|
||||
}>;
|
||||
|
||||
type NormalizedStorageCommand = ArtifactAuthority &
|
||||
Readonly<{
|
||||
byteLength: number;
|
||||
@@ -127,7 +137,9 @@ const DIAGNOSTIC_CONTEXT = Object.freeze({
|
||||
});
|
||||
|
||||
function configurationError(message: string): TypeError {
|
||||
return new TypeError(`S3 Remote Worker Artifact store is invalid: ${message}`);
|
||||
return new TypeError(
|
||||
`S3 Remote Worker Artifact store is invalid: ${message}`,
|
||||
);
|
||||
}
|
||||
|
||||
function prepareOptions(
|
||||
@@ -175,14 +187,15 @@ function prepareOptions(
|
||||
throw configurationError('expected bucket owner is invalid');
|
||||
}
|
||||
const encryption = options.encryption;
|
||||
if (!encryption || typeof encryption !== 'object' || Array.isArray(encryption)) {
|
||||
if (
|
||||
!encryption ||
|
||||
typeof encryption !== 'object' ||
|
||||
Array.isArray(encryption)
|
||||
) {
|
||||
throw configurationError('encryption is required');
|
||||
}
|
||||
let preparedEncryption: PreparedOptions['encryption'];
|
||||
if (
|
||||
encryption.mode === 's3' &&
|
||||
Object.keys(encryption).length === 1
|
||||
) {
|
||||
if (encryption.mode === 's3' && Object.keys(encryption).length === 1) {
|
||||
preparedEncryption = Object.freeze({
|
||||
ServerSideEncryption: ServerSideEncryption.AES256,
|
||||
});
|
||||
@@ -383,9 +396,8 @@ function finalMetadata(
|
||||
),
|
||||
'ql3-byte-length': String(command.byteLength),
|
||||
'ql3-content-sha256': sha256,
|
||||
'ql3-truncated': command.truncated === undefined
|
||||
? 'omitted'
|
||||
: String(command.truncated),
|
||||
'ql3-truncated':
|
||||
command.truncated === undefined ? 'omitted' : String(command.truncated),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -460,6 +472,99 @@ function parseStoredReceipt(
|
||||
);
|
||||
}
|
||||
|
||||
function canonicalETag(value: unknown): string {
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
value.length < 3 ||
|
||||
value.length > 256 ||
|
||||
!/^"[^"\u0000-\u001f\u007f]+"$/.test(value)
|
||||
) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function assertRangeMetadata(
|
||||
authority: ArtifactAuthority,
|
||||
receipt: Readonly<RemoteWorkerArtifactReceipt>,
|
||||
output: Readonly<{
|
||||
ContentLength?: number | undefined;
|
||||
ContentRange?: string | undefined;
|
||||
ContentType?: string | undefined;
|
||||
ETag?: string | undefined;
|
||||
Metadata?: Readonly<Record<string, string | undefined>> | undefined;
|
||||
}>,
|
||||
eTag: string,
|
||||
start: number,
|
||||
endExclusive: number,
|
||||
): void {
|
||||
const metadata = output.Metadata;
|
||||
const truncated =
|
||||
receipt.truncated === undefined ? 'omitted' : String(receipt.truncated);
|
||||
if (
|
||||
output.ContentLength !== endExclusive - start ||
|
||||
output.ContentRange !==
|
||||
`bytes ${start}-${endExclusive - 1}/${receipt.byteLength}` ||
|
||||
output.ContentType !== REMOTE_WORKER_ARTIFACT_CONTENT_TYPE ||
|
||||
canonicalETag(output.ETag) !== eTag ||
|
||||
metadataValue(metadata, 'ql3-schema') !== METADATA_SCHEMA ||
|
||||
metadataValue(metadata, 'ql3-project-sha256') !==
|
||||
fieldDigest('project', authority.projectId) ||
|
||||
metadataValue(metadata, 'ql3-run-sha256') !==
|
||||
fieldDigest('run', authority.runId) ||
|
||||
metadataValue(metadata, 'ql3-attempt-sha256') !==
|
||||
fieldDigest('attempt', authority.attemptId) ||
|
||||
metadataValue(metadata, 'ql3-log-artifact-sha256') !==
|
||||
fieldDigest('log-artifact', authority.logArtifactId) ||
|
||||
metadataValue(metadata, 'ql3-byte-length') !== String(receipt.byteLength) ||
|
||||
metadataValue(metadata, 'ql3-content-sha256') !== receipt.sha256 ||
|
||||
metadataValue(metadata, 'ql3-truncated') !== truncated
|
||||
) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
}
|
||||
|
||||
async function readBoundedRangeBody(
|
||||
body: unknown,
|
||||
expectedBytes: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Buffer> {
|
||||
if (
|
||||
!body ||
|
||||
typeof body !== 'object' ||
|
||||
!(Symbol.asyncIterator in body) ||
|
||||
typeof (body as AsyncIterable<Uint8Array>)[Symbol.asyncIterator] !==
|
||||
'function'
|
||||
) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
const content = Buffer.allocUnsafe(expectedBytes);
|
||||
let received = 0;
|
||||
try {
|
||||
for await (const chunk of body as AsyncIterable<Uint8Array>) {
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
if (!(chunk instanceof Uint8Array) || chunk.byteLength === 0) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
if (received + chunk.byteLength > expectedBytes) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength).copy(
|
||||
content,
|
||||
received,
|
||||
);
|
||||
received += chunk.byteLength;
|
||||
}
|
||||
if (received !== expectedBytes) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
return content;
|
||||
} catch (error) {
|
||||
content.fill(0);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function isNotFound(error: unknown): boolean {
|
||||
if (!error || typeof error !== 'object') return false;
|
||||
const value = error as {
|
||||
@@ -467,13 +572,17 @@ function isNotFound(error: unknown): boolean {
|
||||
Code?: unknown;
|
||||
$metadata?: { httpStatusCode?: unknown };
|
||||
};
|
||||
return value.name === 'NotFound' ||
|
||||
return (
|
||||
value.name === 'NotFound' ||
|
||||
value.name === 'NoSuchKey' ||
|
||||
value.Code === 'NoSuchKey' ||
|
||||
value.$metadata?.httpStatusCode === 404;
|
||||
value.$metadata?.httpStatusCode === 404
|
||||
);
|
||||
}
|
||||
|
||||
function requestOptions(signal?: AbortSignal): { abortSignal: AbortSignal } | undefined {
|
||||
function requestOptions(
|
||||
signal?: AbortSignal,
|
||||
): { abortSignal: AbortSignal } | undefined {
|
||||
return signal === undefined ? undefined : { abortSignal: signal };
|
||||
}
|
||||
|
||||
@@ -549,7 +658,8 @@ class ArtifactContentDigest {
|
||||
* objects are never overwritten or deleted by this adapter.
|
||||
*/
|
||||
export class S3ClusterRemoteWorkerArtifactStore
|
||||
implements ClusterRemoteWorkerArtifactStore {
|
||||
implements ClusterRemoteWorkerArtifactStore
|
||||
{
|
||||
private readonly options: PreparedOptions;
|
||||
|
||||
constructor(options: S3ClusterRemoteWorkerArtifactStoreOptions) {
|
||||
@@ -561,6 +671,90 @@ export class S3ClusterRemoteWorkerArtifactStore
|
||||
signal?: AbortSignal,
|
||||
): Promise<Readonly<RemoteWorkerArtifactReceipt> | undefined> {
|
||||
const authority = normalizeLookup(lookup);
|
||||
return (await this.head(authority, signal))?.receipt;
|
||||
}
|
||||
|
||||
async readLogRange(
|
||||
rawIdentity: Readonly<RunAttemptLogReadIdentity>,
|
||||
rawRange: Readonly<RunAttemptLogReadRange>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<RunAttemptLogRangeReadResult> {
|
||||
const authority = normalizeLookup(rawIdentity);
|
||||
const range = normalizeRunAttemptLogReadRange(rawRange);
|
||||
const stored = await this.head(authority, signal);
|
||||
if (!stored) return Object.freeze({ status: 'missing' as const });
|
||||
const start = Math.min(range.offset, stored.receipt.byteLength);
|
||||
const endExclusive = Math.min(
|
||||
start + range.length,
|
||||
stored.receipt.byteLength,
|
||||
);
|
||||
const truncation = Object.freeze({
|
||||
truncated: stored.receipt.truncated ?? ('unknown' as const),
|
||||
});
|
||||
if (start === endExclusive) {
|
||||
return Object.freeze({
|
||||
status: 'available' as const,
|
||||
content: Buffer.alloc(0),
|
||||
start,
|
||||
endExclusive,
|
||||
totalBytes: stored.receipt.byteLength,
|
||||
truncation,
|
||||
});
|
||||
}
|
||||
const eTag = canonicalETag(stored.eTag);
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
try {
|
||||
const output = await this.options.client.send(
|
||||
new GetObjectCommand({
|
||||
Bucket: this.options.bucket,
|
||||
Key: finalObjectKey(this.options.prefix, authority),
|
||||
IfMatch: eTag,
|
||||
Range: `bytes=${start}-${endExclusive - 1}`,
|
||||
...(this.options.expectedBucketOwner === undefined
|
||||
? {}
|
||||
: { ExpectedBucketOwner: this.options.expectedBucketOwner }),
|
||||
}),
|
||||
requestOptions(signal),
|
||||
);
|
||||
assertRangeMetadata(
|
||||
authority,
|
||||
stored.receipt,
|
||||
output,
|
||||
eTag,
|
||||
start,
|
||||
endExclusive,
|
||||
);
|
||||
const bytes = await readBoundedRangeBody(
|
||||
output.Body,
|
||||
endExclusive - start,
|
||||
signal,
|
||||
);
|
||||
return Object.freeze({
|
||||
status: 'available' as const,
|
||||
content: bytes,
|
||||
start,
|
||||
endExclusive,
|
||||
totalBytes: stored.receipt.byteLength,
|
||||
...(endExclusive < stored.receipt.byteLength
|
||||
? { nextOffset: endExclusive }
|
||||
: {}),
|
||||
truncation,
|
||||
});
|
||||
} catch (error) {
|
||||
if (isNotFound(error)) {
|
||||
return Object.freeze({ status: 'missing' as const });
|
||||
}
|
||||
if (error instanceof S3ClusterRemoteWorkerArtifactStoreError) throw error;
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('unavailable', {
|
||||
cause: error,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private async head(
|
||||
authority: ArtifactAuthority,
|
||||
signal?: AbortSignal,
|
||||
): Promise<StoredArtifactHead | undefined> {
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
try {
|
||||
const output = await this.options.client.send(
|
||||
@@ -574,7 +768,10 @@ export class S3ClusterRemoteWorkerArtifactStore
|
||||
}),
|
||||
requestOptions(signal),
|
||||
);
|
||||
return parseStoredReceipt(authority, output);
|
||||
return Object.freeze({
|
||||
receipt: parseStoredReceipt(authority, output),
|
||||
...(output.ETag === undefined ? {} : { eTag: output.ETag }),
|
||||
});
|
||||
} catch (error) {
|
||||
if (isNotFound(error)) return undefined;
|
||||
if (error instanceof S3ClusterRemoteWorkerArtifactStoreError) throw error;
|
||||
@@ -604,9 +801,7 @@ export class S3ClusterRemoteWorkerArtifactStore
|
||||
existing.truncated !== command.truncated ||
|
||||
existing.sha256 !== incomingSha256
|
||||
) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError(
|
||||
'integrity_mismatch',
|
||||
);
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
return existing;
|
||||
}
|
||||
@@ -693,13 +888,11 @@ export class S3ClusterRemoteWorkerArtifactStore
|
||||
stored.truncated !== command.truncated ||
|
||||
stored.sha256 !== sha256
|
||||
) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError(
|
||||
'integrity_mismatch',
|
||||
);
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
result = Object.freeze({
|
||||
...stored,
|
||||
status: copied ? 'stored' as const : 'already_stored' as const,
|
||||
status: copied ? ('stored' as const) : ('already_stored' as const),
|
||||
});
|
||||
} catch (error) {
|
||||
primaryError = error;
|
||||
@@ -770,9 +963,7 @@ export class S3ClusterRemoteWorkerArtifactStore
|
||||
output.Metadata?.['ql3-owner-sha256'] !== ownerSha256 ||
|
||||
canonicalChecksum(output.ChecksumSHA256) !== sha256
|
||||
) {
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError(
|
||||
'integrity_mismatch',
|
||||
);
|
||||
throw new S3ClusterRemoteWorkerArtifactStoreError('integrity_mismatch');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -16,6 +16,11 @@ import {
|
||||
type RemoteWorkerCompletionRepository,
|
||||
type RemoteWorkerCompletionResult,
|
||||
} from '@qinglong/runtime-core/remote-worker-completion';
|
||||
import type {
|
||||
RunAttemptLogRangeReadResult,
|
||||
RunAttemptLogReadIdentity,
|
||||
RunAttemptLogReadRange,
|
||||
} from '@qinglong/runtime-core/run-attempt-log-read';
|
||||
|
||||
export interface ClusterRemoteWorkerArtifactStorageCommand {
|
||||
readonly projectId: string;
|
||||
@@ -47,6 +52,12 @@ export interface ClusterRemoteWorkerArtifactStore {
|
||||
lookup: Readonly<ClusterRemoteWorkerArtifactLookup>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Readonly<RemoteWorkerArtifactReceipt> | undefined>;
|
||||
/** Optional during Alpha so upload-only test and alternate stores remain compatible. */
|
||||
readLogRange?(
|
||||
identity: Readonly<RunAttemptLogReadIdentity>,
|
||||
range: Readonly<RunAttemptLogReadRange>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<RunAttemptLogRangeReadResult>;
|
||||
}
|
||||
|
||||
export interface ClusterRemoteWorkerArtifactUploadInput {
|
||||
@@ -353,11 +364,13 @@ export class ClusterRemoteWorkerCompletionService {
|
||||
}
|
||||
try {
|
||||
const result = normalizeRemoteWorkerCompletionResult(
|
||||
await this.repository.complete(Object.freeze({
|
||||
...command,
|
||||
attemptEventId: eventId(this.createEventId),
|
||||
runEventId: eventId(this.createEventId),
|
||||
})),
|
||||
await this.repository.complete(
|
||||
Object.freeze({
|
||||
...command,
|
||||
attemptEventId: eventId(this.createEventId),
|
||||
runEventId: eventId(this.createEventId),
|
||||
}),
|
||||
),
|
||||
);
|
||||
if (
|
||||
result.runId !== command.runId ||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
// Remote execution owns the least-privilege assembly of Worker-facing runtime capabilities.
|
||||
import type { PostgresPool } from '@qinglong/runtime-core';
|
||||
import type { RemoteWorkerSecretValueProvider } from '@qinglong/runtime-core/remote-secret-delivery';
|
||||
import type { RunAttemptLogRangeReader } from '@qinglong/runtime-core/run-attempt-log-read';
|
||||
import {
|
||||
PostgresClusterDispatchSource,
|
||||
PostgresRemoteRunActivationRepository,
|
||||
@@ -11,23 +12,15 @@ import {
|
||||
PostgresTaskExecutionRevisionSource,
|
||||
PostgresWorkerSessionRepository,
|
||||
} from '@qinglong/cluster-postgres/runtime';
|
||||
import {
|
||||
ClusterRemoteWorkerOfferClaimService,
|
||||
} from './remoteWorkerDispatcher';
|
||||
import {
|
||||
ClusterRemoteRunActivationService,
|
||||
} from './remoteRunActivationService';
|
||||
import {
|
||||
ClusterRemoteWorkerSecretDeliveryService,
|
||||
} from './remoteWorkerSecretDeliveryService';
|
||||
import { ClusterRemoteWorkerOfferClaimService } from './remoteWorkerDispatcher';
|
||||
import { ClusterRemoteRunActivationService } from './remoteRunActivationService';
|
||||
import { ClusterRemoteWorkerSecretDeliveryService } from './remoteWorkerSecretDeliveryService';
|
||||
import {
|
||||
ClusterRemoteWorkerArtifactService,
|
||||
ClusterRemoteWorkerCompletionService,
|
||||
type ClusterRemoteWorkerArtifactStore,
|
||||
} from './remoteWorkerCompletionService';
|
||||
import {
|
||||
ClusterRemoteWorkerLeaseControlService,
|
||||
} from './remoteWorkerLeaseControlService';
|
||||
import { ClusterRemoteWorkerLeaseControlService } from './remoteWorkerLeaseControlService';
|
||||
import type { WorkerIngressPipelineOptions } from '../worker-ingress/workerIngressPipeline';
|
||||
|
||||
export interface ClusterWorkerRuntimeDependencies {
|
||||
@@ -49,6 +42,7 @@ export interface ClusterWorkerRuntimePort {
|
||||
readonly leaseControl: NonNullable<
|
||||
WorkerIngressPipelineOptions['leaseControl']
|
||||
>;
|
||||
readonly runAttemptLogRead?: RunAttemptLogRangeReader;
|
||||
}
|
||||
|
||||
export function createClusterWorkerRuntimePort(
|
||||
@@ -67,9 +61,11 @@ export function createClusterWorkerRuntimePort(
|
||||
}
|
||||
|
||||
const workerSessions = new PostgresWorkerSessionRepository(pool);
|
||||
const completionRepository =
|
||||
new PostgresRemoteWorkerCompletionRepository(pool);
|
||||
const completionRepository = new PostgresRemoteWorkerCompletionRepository(
|
||||
pool,
|
||||
);
|
||||
const secretProvider = dependencies.secretProvider;
|
||||
const readLogRange = dependencies.artifactStore.readLogRange;
|
||||
return Object.freeze({
|
||||
offers: new ClusterRemoteWorkerOfferClaimService(
|
||||
new PostgresClusterDispatchSource(pool),
|
||||
@@ -99,5 +95,12 @@ export function createClusterWorkerRuntimePort(
|
||||
leaseControl: new ClusterRemoteWorkerLeaseControlService(
|
||||
new PostgresRemoteWorkerLeaseControlRepository(pool),
|
||||
),
|
||||
...(readLogRange === undefined
|
||||
? {}
|
||||
: {
|
||||
runAttemptLogRead: Object.freeze({
|
||||
read: readLogRange.bind(dependencies.artifactStore),
|
||||
}),
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
import {
|
||||
InvalidRunAttemptLogReadError,
|
||||
RunAttemptLogReadService,
|
||||
RunAttemptLogReadUnavailableError,
|
||||
type RunAttemptLogRangeReader,
|
||||
type RunAttemptLogReadResult,
|
||||
} from '@qinglong/runtime-core/run-attempt-log-read';
|
||||
import type { RunRepositoryReader } from '@qinglong/runtime-core/run-repository';
|
||||
|
||||
import type { ClusterControlAdmissionResponse } from '../transport/httpSurface';
|
||||
import type {
|
||||
ClusterControlAuthorizedOperationRequest,
|
||||
ClusterControlRouteDefinition,
|
||||
ClusterControlRouteParameters,
|
||||
} from '../transport/routeRegistry';
|
||||
|
||||
export const CLUSTER_CONTROL_RUN_ATTEMPT_LOG_READ_ROUTE = Object.freeze({
|
||||
method: 'GET' as const,
|
||||
path: '/api/v3/projects/{projectId}/runs/{runId}/attempts/{attemptId}/log',
|
||||
operationId: 'run.log.read',
|
||||
permission: 'artifact.read',
|
||||
projectParameter: 'projectId',
|
||||
allowedQuery: Object.freeze(['length', 'offset']),
|
||||
});
|
||||
|
||||
const DEFAULT_READ_BYTES = 64 * 1024;
|
||||
const MAXIMUM_READ_BYTES = 256 * 1024;
|
||||
|
||||
function response(
|
||||
statusCode: number,
|
||||
body: Readonly<Record<string, unknown>>,
|
||||
): ClusterControlAdmissionResponse {
|
||||
return Object.freeze({ statusCode, body: Object.freeze(body) });
|
||||
}
|
||||
|
||||
function parseQuery(
|
||||
query: Readonly<Record<string, readonly string[]>>,
|
||||
): Readonly<{ offset: number; length: number }> {
|
||||
const offsetValues = query.offset;
|
||||
const lengthValues = query.length;
|
||||
if (
|
||||
(offsetValues !== undefined && offsetValues.length !== 1) ||
|
||||
(lengthValues !== undefined && lengthValues.length !== 1)
|
||||
) {
|
||||
throw new TypeError();
|
||||
}
|
||||
const rawOffset = offsetValues?.[0];
|
||||
const offset = rawOffset === undefined ? 0 : Number(rawOffset);
|
||||
const rawLength = lengthValues?.[0];
|
||||
const length =
|
||||
rawLength === undefined ? DEFAULT_READ_BYTES : Number(rawLength);
|
||||
if (
|
||||
!Number.isSafeInteger(offset) ||
|
||||
offset < 0 ||
|
||||
(rawOffset !== undefined && String(offset) !== rawOffset) ||
|
||||
!Number.isSafeInteger(length) ||
|
||||
length < 1 ||
|
||||
length > MAXIMUM_READ_BYTES ||
|
||||
(rawLength !== undefined && String(length) !== rawLength)
|
||||
) {
|
||||
throw new TypeError();
|
||||
}
|
||||
return Object.freeze({ offset, length });
|
||||
}
|
||||
|
||||
function validateQuery(
|
||||
query: Readonly<Record<string, readonly string[]>>,
|
||||
): void {
|
||||
parseQuery(query);
|
||||
}
|
||||
|
||||
function projection(
|
||||
result: Extract<RunAttemptLogReadResult, { readonly status: 'available' }>,
|
||||
): Readonly<Record<string, unknown>> {
|
||||
return Object.freeze({
|
||||
schema: 'qinglong/run-attempt-log-read-result@v1',
|
||||
status: 'available',
|
||||
projectId: result.projectId,
|
||||
runId: result.runId,
|
||||
attemptId: result.attemptId,
|
||||
range: Object.freeze({
|
||||
start: result.start,
|
||||
endExclusive: result.endExclusive,
|
||||
totalBytes: result.totalBytes,
|
||||
...(result.nextOffset === undefined
|
||||
? {}
|
||||
: { nextOffset: result.nextOffset }),
|
||||
}),
|
||||
encoding: 'base64',
|
||||
content: Buffer.from(
|
||||
result.content.buffer,
|
||||
result.content.byteOffset,
|
||||
result.content.byteLength,
|
||||
).toString('base64'),
|
||||
truncation: result.truncation,
|
||||
});
|
||||
}
|
||||
|
||||
export function createClusterControlRunAttemptLogReadRoute(
|
||||
runs: Pick<RunRepositoryReader, 'findRunById' | 'findAttemptById'>,
|
||||
reader?: RunAttemptLogRangeReader,
|
||||
): Readonly<ClusterControlRouteDefinition> {
|
||||
if (
|
||||
!runs ||
|
||||
typeof runs.findRunById !== 'function' ||
|
||||
typeof runs.findAttemptById !== 'function' ||
|
||||
(reader !== undefined && typeof reader.read !== 'function')
|
||||
) {
|
||||
throw new TypeError(
|
||||
'Cluster-control Run Attempt log read dependencies are invalid',
|
||||
);
|
||||
}
|
||||
const service =
|
||||
reader === undefined
|
||||
? undefined
|
||||
: new RunAttemptLogReadService(runs, reader, {
|
||||
executorType: 'remote_worker',
|
||||
artifactIdPattern: /^wlog-[a-f0-9]{30}$/,
|
||||
maximumReadBytes: MAXIMUM_READ_BYTES,
|
||||
activeMissingIsPending: true,
|
||||
});
|
||||
return Object.freeze({
|
||||
...CLUSTER_CONTROL_RUN_ATTEMPT_LOG_READ_ROUTE,
|
||||
validateQuery,
|
||||
async handle(
|
||||
authorized: ClusterControlAuthorizedOperationRequest,
|
||||
parameters: ClusterControlRouteParameters,
|
||||
) {
|
||||
if (authorized.request.body !== null) {
|
||||
return response(400, { code: 'invalid_request_body' });
|
||||
}
|
||||
if (authorized.projectId === null || service === undefined) {
|
||||
return response(503, { code: 'artifact_unavailable' });
|
||||
}
|
||||
let range;
|
||||
try {
|
||||
range = parseQuery(authorized.request.query);
|
||||
} catch {
|
||||
return response(400, { code: 'invalid_run_log_read_query' });
|
||||
}
|
||||
try {
|
||||
const result = await service.read({
|
||||
projectId: authorized.projectId,
|
||||
runId: parameters.runId!,
|
||||
attemptId: parameters.attemptId!,
|
||||
range,
|
||||
signal: authorized.request.signal,
|
||||
});
|
||||
if (result.status === 'not_found') {
|
||||
return response(404, { code: 'artifact_not_found' });
|
||||
}
|
||||
if (result.status === 'pending') {
|
||||
return response(202, {
|
||||
schema: 'qinglong/run-attempt-log-read-result@v1',
|
||||
status: 'pending',
|
||||
projectId: result.projectId,
|
||||
runId: result.runId,
|
||||
attemptId: result.attemptId,
|
||||
});
|
||||
}
|
||||
if (result.status === 'missing') {
|
||||
return response(503, { code: 'artifact_unavailable' });
|
||||
}
|
||||
return response(200, projection(result));
|
||||
} catch (error) {
|
||||
if (error instanceof InvalidRunAttemptLogReadError) {
|
||||
return response(400, { code: 'invalid_run_log_read_request' });
|
||||
}
|
||||
if (error instanceof RunAttemptLogReadUnavailableError) {
|
||||
return response(503, { code: 'artifact_unavailable' });
|
||||
}
|
||||
return response(503, { code: 'artifact_unavailable' });
|
||||
}
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -308,6 +308,13 @@ export function createClusterControlAdmissionPipeline(
|
||||
now,
|
||||
);
|
||||
if (decision.effect === 'deny') {
|
||||
if (route.permission === 'artifact.read') {
|
||||
throw securityError(
|
||||
404,
|
||||
'artifact_not_found',
|
||||
'Cluster-control Artifact is not available',
|
||||
);
|
||||
}
|
||||
throw securityError(
|
||||
403,
|
||||
'forbidden',
|
||||
@@ -315,6 +322,13 @@ export function createClusterControlAdmissionPipeline(
|
||||
);
|
||||
}
|
||||
if (decision.effect === 'require_approval') {
|
||||
if (route.permission === 'artifact.read') {
|
||||
throw securityError(
|
||||
404,
|
||||
'artifact_not_found',
|
||||
'Cluster-control Artifact is not available',
|
||||
);
|
||||
}
|
||||
throw securityError(
|
||||
403,
|
||||
'approval_required',
|
||||
|
||||
@@ -573,6 +573,9 @@ test('injects reviewed Worker operations without exposing the runtime Pool', asy
|
||||
async inspect() {
|
||||
throw new Error('not invoked during assembly');
|
||||
},
|
||||
async readLogRange() {
|
||||
throw new Error('not invoked during assembly');
|
||||
},
|
||||
};
|
||||
const result = await bootstrapClusterControlRuntime(
|
||||
bootstrapOptions(events, {
|
||||
@@ -596,6 +599,10 @@ test('injects reviewed Worker operations without exposing the runtime Pool', asy
|
||||
typeof input.workerRuntime.leaseControl.control,
|
||||
'function',
|
||||
);
|
||||
assert.equal(
|
||||
typeof input.workerRuntime.runAttemptLogRead.read,
|
||||
'function',
|
||||
);
|
||||
return activationStack(events);
|
||||
},
|
||||
}),
|
||||
|
||||
@@ -143,6 +143,9 @@ function fixture(overrides = {}) {
|
||||
},
|
||||
];
|
||||
},
|
||||
async findAttemptById() {
|
||||
return null;
|
||||
},
|
||||
},
|
||||
trustedToolStorage: {
|
||||
stepRuns: {
|
||||
@@ -190,7 +193,8 @@ function fixture(overrides = {}) {
|
||||
taskDefinitions: {
|
||||
async findCurrentTaskDefinition(projectId, taskId) {
|
||||
events.push(`task-get:${projectId}:${taskId}`);
|
||||
return projectId === currentTask.projectId && taskId === currentTask.taskId
|
||||
return projectId === currentTask.projectId &&
|
||||
taskId === currentTask.taskId
|
||||
? currentTask
|
||||
: null;
|
||||
},
|
||||
@@ -328,6 +332,7 @@ test('production composition exposes the reviewed Run and Workflow routes', asyn
|
||||
'run.list',
|
||||
'run.events.list',
|
||||
'run.steps.list',
|
||||
'run.log.read',
|
||||
'run.cancel',
|
||||
'workflow.read',
|
||||
'workflow.run.read',
|
||||
@@ -401,6 +406,15 @@ test('production composition exposes the reviewed Run and Workflow routes', asyn
|
||||
body: { steps: [], hasMore: false, next: null },
|
||||
});
|
||||
|
||||
const log = await invoke(
|
||||
stack,
|
||||
metadata('/api/v3/projects/project-1/runs/run-1/attempts/attempt-1/log'),
|
||||
);
|
||||
assert.deepEqual(log, {
|
||||
statusCode: 503,
|
||||
body: { code: 'artifact_unavailable' },
|
||||
});
|
||||
|
||||
const cancellation = await invoke(
|
||||
stack,
|
||||
metadata('/api/v3/projects/project-1/runs/run-1/cancellation', 'POST', {
|
||||
@@ -414,6 +428,7 @@ test('production composition exposes the reviewed Run and Workflow routes', asyn
|
||||
assert.equal(events.includes('audit:run.get:allowed'), true);
|
||||
assert.equal(events.includes('audit:run.events.list:allowed'), true);
|
||||
assert.equal(events.includes('audit:run.steps.list:allowed'), true);
|
||||
assert.equal(events.includes('audit:run.log.read:allowed'), true);
|
||||
assert.equal(events.includes('audit:run.cancel:allowed'), true);
|
||||
|
||||
const workflows = await invoke(
|
||||
@@ -539,6 +554,69 @@ test('production composition fails closed for an unreviewed route', async () =>
|
||||
);
|
||||
});
|
||||
|
||||
test('wires the production Worker object reader into the Project-scoped log route', async () => {
|
||||
const { input } = fixture();
|
||||
const run = await input.runs.findRunById('run-1');
|
||||
const logArtifactId = `wlog-${'a'.repeat(30)}`;
|
||||
const stack = createProductionClusterControlApplicationStack({
|
||||
...input,
|
||||
runs: {
|
||||
...input.runs,
|
||||
async findRunById() {
|
||||
return { ...run, status: 'running' };
|
||||
},
|
||||
async findAttemptById() {
|
||||
return {
|
||||
id: 'attempt-1',
|
||||
runId: 'run-1',
|
||||
attempt: 1,
|
||||
status: 'running',
|
||||
executorType: 'remote_worker',
|
||||
logArtifactId,
|
||||
callbackSequence: 0,
|
||||
createdAtMs: 1,
|
||||
};
|
||||
},
|
||||
},
|
||||
workerRuntime: {
|
||||
offers: { claimNext() {} },
|
||||
activation: {
|
||||
acknowledgeStarting() {},
|
||||
acknowledgeRunning() {},
|
||||
failStart() {},
|
||||
},
|
||||
artifacts: { upload() {} },
|
||||
completion: { complete() {} },
|
||||
leaseControl: { control() {} },
|
||||
runAttemptLogRead: {
|
||||
async read(identity, range) {
|
||||
assert.equal(identity.logArtifactId, logArtifactId);
|
||||
assert.deepEqual(range, { offset: 1, length: 4 });
|
||||
return {
|
||||
status: 'available',
|
||||
content: Buffer.from('prod'),
|
||||
start: 1,
|
||||
endExclusive: 5,
|
||||
totalBytes: 5,
|
||||
truncation: { truncated: false },
|
||||
};
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = await invoke(
|
||||
stack,
|
||||
metadata(
|
||||
'/api/v3/projects/project-1/runs/run-1/attempts/attempt-1/log',
|
||||
'GET',
|
||||
null,
|
||||
{ offset: ['1'], length: ['4'] },
|
||||
),
|
||||
);
|
||||
assert.equal(result.statusCode, 200);
|
||||
assert.equal(Buffer.from(result.body.content, 'base64').toString(), 'prod');
|
||||
});
|
||||
|
||||
test('optionally exposes Prompt execution behind shared admission and policy', async () => {
|
||||
const { events, input } = fixture();
|
||||
let command;
|
||||
|
||||
@@ -0,0 +1,251 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const { test } = require('node:test');
|
||||
|
||||
const {
|
||||
createClusterControlAdmissionPipeline,
|
||||
} = require('@qinglong/cluster-control/admission');
|
||||
const {
|
||||
createClusterControlRouteRegistry,
|
||||
} = require('@qinglong/cluster-control/routes');
|
||||
const {
|
||||
CLUSTER_CONTROL_RUN_ATTEMPT_LOG_READ_ROUTE,
|
||||
createClusterControlRunAttemptLogReadRoute,
|
||||
} = require('../dist/run/runAttemptLogReadRoute.js');
|
||||
|
||||
const PRINCIPAL = Object.freeze({
|
||||
subject: Object.freeze({ type: 'user', id: 'usr_viewer' }),
|
||||
authenticationId: 'session:viewer',
|
||||
authenticatedAtMs: 9_000,
|
||||
expiresAtMs: 11_000,
|
||||
assurance: 'single_factor',
|
||||
});
|
||||
|
||||
function run(overrides = {}) {
|
||||
return {
|
||||
id: 'run_123',
|
||||
projectId: 'prj_default',
|
||||
taskId: 'task_1',
|
||||
taskRevision: 'revision_1',
|
||||
triggerType: 'task_start',
|
||||
executionOrigin: 'manual',
|
||||
executionOwner: 'runtime',
|
||||
status: 'running',
|
||||
version: 2,
|
||||
eventSequence: 2,
|
||||
priority: 0,
|
||||
createdAtMs: 1,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function attempt(overrides = {}) {
|
||||
return {
|
||||
id: 'attempt_123',
|
||||
runId: 'run_123',
|
||||
attempt: 1,
|
||||
status: 'running',
|
||||
executorType: 'remote_worker',
|
||||
logArtifactId: `wlog-${'a'.repeat(30)}`,
|
||||
callbackSequence: 0,
|
||||
createdAtMs: 1,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function metadata(query = {}) {
|
||||
return Object.freeze({
|
||||
requestId: 'request-log-read',
|
||||
method: 'GET',
|
||||
path: '/api/v3/projects/prj_default/runs/run_123/attempts/attempt_123/log',
|
||||
query: Object.freeze(query),
|
||||
headers: Object.freeze({ authorization: 'Bearer opaque' }),
|
||||
signal: new AbortController().signal,
|
||||
});
|
||||
}
|
||||
|
||||
function pipeline(options = {}) {
|
||||
const events = options.events ?? [];
|
||||
const repository = options.repository ?? {
|
||||
async findRunById() {
|
||||
events.push('run');
|
||||
return run();
|
||||
},
|
||||
async findAttemptById() {
|
||||
events.push('attempt');
|
||||
return attempt();
|
||||
},
|
||||
};
|
||||
const reader = options.reader ?? {
|
||||
async read(identity, range) {
|
||||
events.push(`storage:${range.offset}:${range.length}`);
|
||||
return {
|
||||
status: 'available',
|
||||
content: Buffer.from('cluster-log'),
|
||||
start: range.offset,
|
||||
endExclusive: range.offset + 11,
|
||||
totalBytes: range.offset + 20,
|
||||
nextOffset: range.offset + 11,
|
||||
truncation: { truncated: false },
|
||||
};
|
||||
},
|
||||
};
|
||||
return createClusterControlAdmissionPipeline({
|
||||
routes: createClusterControlRouteRegistry([
|
||||
createClusterControlRunAttemptLogReadRoute(repository, reader),
|
||||
]),
|
||||
authenticator: {
|
||||
authenticate() {
|
||||
events.push('authenticate');
|
||||
return PRINCIPAL;
|
||||
},
|
||||
},
|
||||
policy: {
|
||||
authorize(request) {
|
||||
events.push(`authorize:${request.permission}`);
|
||||
return options.effect
|
||||
? { effect: options.effect, reasons: ['masked'], fence: null }
|
||||
: {
|
||||
effect: 'allow',
|
||||
reasons: ['role_grant'],
|
||||
fence: { projectVersion: 2, bindingVersion: 3 },
|
||||
};
|
||||
},
|
||||
},
|
||||
audit: {
|
||||
record(record) {
|
||||
events.push(`audit:${record.outcome}`);
|
||||
},
|
||||
},
|
||||
now: () => 10_000,
|
||||
});
|
||||
}
|
||||
|
||||
async function invoke(value, query = {}) {
|
||||
const prepared = await value.prepare(metadata(query));
|
||||
return prepared.handle(null);
|
||||
}
|
||||
|
||||
test('publishes the immutable Artifact-scoped route contract', () => {
|
||||
assert.deepEqual(CLUSTER_CONTROL_RUN_ATTEMPT_LOG_READ_ROUTE, {
|
||||
method: 'GET',
|
||||
path: '/api/v3/projects/{projectId}/runs/{runId}/attempts/{attemptId}/log',
|
||||
operationId: 'run.log.read',
|
||||
permission: 'artifact.read',
|
||||
projectParameter: 'projectId',
|
||||
allowedQuery: ['length', 'offset'],
|
||||
});
|
||||
});
|
||||
|
||||
test('authorizes and audits before metadata and one bounded range read', async () => {
|
||||
const events = [];
|
||||
const result = await invoke(pipeline({ events }), {
|
||||
offset: ['4'],
|
||||
length: ['16'],
|
||||
});
|
||||
assert.equal(result.statusCode, 200);
|
||||
assert.equal(result.body.schema, 'qinglong/run-attempt-log-read-result@v1');
|
||||
assert.equal(
|
||||
Buffer.from(result.body.content, 'base64').toString(),
|
||||
'cluster-log',
|
||||
);
|
||||
assert.deepEqual(result.body.range, {
|
||||
start: 4,
|
||||
endExclusive: 15,
|
||||
totalBytes: 24,
|
||||
nextOffset: 15,
|
||||
});
|
||||
assert.deepEqual(events, [
|
||||
'authenticate',
|
||||
'authorize:artifact.read',
|
||||
'audit:allowed',
|
||||
'run',
|
||||
'attempt',
|
||||
'storage:4:16',
|
||||
]);
|
||||
});
|
||||
|
||||
test('uses the Cluster default window and rejects unbounded query values', async () => {
|
||||
const events = [];
|
||||
assert.equal((await invoke(pipeline({ events }))).statusCode, 200);
|
||||
assert.equal(events.at(-1), `storage:0:${64 * 1024}`);
|
||||
for (const query of [
|
||||
{ offset: ['-1'] },
|
||||
{ offset: ['04'] },
|
||||
{ length: ['0'] },
|
||||
{ length: [String(256 * 1024 + 1)] },
|
||||
{ length: ['1', '2'] },
|
||||
]) {
|
||||
await assert.rejects(
|
||||
pipeline().prepare(metadata(query)),
|
||||
(error) =>
|
||||
error.statusCode === 400 && error.code === 'invalid_route_query',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('masks deny and approval without reading Run or object storage', async () => {
|
||||
for (const effect of ['deny', 'require_approval']) {
|
||||
let touched = false;
|
||||
await assert.rejects(
|
||||
pipeline({
|
||||
effect,
|
||||
repository: {
|
||||
async findRunById() {
|
||||
touched = true;
|
||||
return run();
|
||||
},
|
||||
async findAttemptById() {
|
||||
touched = true;
|
||||
return attempt();
|
||||
},
|
||||
},
|
||||
reader: {
|
||||
async read() {
|
||||
touched = true;
|
||||
return { status: 'missing' };
|
||||
},
|
||||
},
|
||||
}).prepare(metadata()),
|
||||
(error) =>
|
||||
error.statusCode === 404 && error.code === 'artifact_not_found',
|
||||
);
|
||||
assert.equal(touched, false);
|
||||
}
|
||||
});
|
||||
|
||||
test('returns pending during upload and fails closed without an object reader', async () => {
|
||||
const pending = pipeline({
|
||||
reader: {
|
||||
async read() {
|
||||
return { status: 'missing' };
|
||||
},
|
||||
},
|
||||
});
|
||||
assert.equal((await invoke(pending)).statusCode, 202);
|
||||
|
||||
const unavailableRoute = createClusterControlRunAttemptLogReadRoute({
|
||||
async findRunById() {
|
||||
return run();
|
||||
},
|
||||
async findAttemptById() {
|
||||
return attempt();
|
||||
},
|
||||
});
|
||||
const prepared = await createClusterControlAdmissionPipeline({
|
||||
routes: createClusterControlRouteRegistry([unavailableRoute]),
|
||||
authenticator: { authenticate: () => PRINCIPAL },
|
||||
policy: {
|
||||
authorize: () => ({
|
||||
effect: 'allow',
|
||||
reasons: ['role_grant'],
|
||||
fence: { projectVersion: 1, bindingVersion: 1 },
|
||||
}),
|
||||
},
|
||||
audit: { record() {} },
|
||||
now: () => 10_000,
|
||||
}).prepare(metadata());
|
||||
assert.deepEqual(await prepared.handle(null), {
|
||||
statusCode: 503,
|
||||
body: { code: 'artifact_unavailable' },
|
||||
});
|
||||
});
|
||||
@@ -19,80 +19,111 @@ const endpoint = process.env.QL3_TEST_S3_ENDPOINT;
|
||||
const accessKeyId = process.env.QL3_TEST_S3_ACCESS_KEY_ID;
|
||||
const secretAccessKey = process.env.QL3_TEST_S3_SECRET_ACCESS_KEY;
|
||||
|
||||
test('real S3-compatible service preserves immutable Artifact evidence', {
|
||||
skip: endpoint && accessKeyId && secretAccessKey
|
||||
? false
|
||||
: 'requires QL3_TEST_S3_ENDPOINT and credentials',
|
||||
}, async () => {
|
||||
const client = new S3Client({
|
||||
endpoint,
|
||||
region: 'us-east-1',
|
||||
forcePathStyle: true,
|
||||
credentials: { accessKeyId, secretAccessKey },
|
||||
});
|
||||
const bucket = `ql3-artifact-${process.pid}-${Date.now()}`.slice(0, 63);
|
||||
const command = Object.freeze({
|
||||
projectId: 'project-s3-integration',
|
||||
runId: 'run-s3-integration',
|
||||
attemptId: 'attempt-s3-integration',
|
||||
logArtifactId: `wlog-${'c'.repeat(30)}`,
|
||||
byteLength: 17,
|
||||
truncated: true,
|
||||
});
|
||||
const content = Buffer.from('real object bytes');
|
||||
const body = (value) => Object.freeze({
|
||||
async *[Symbol.asyncIterator]() {
|
||||
yield value.subarray(0, 4);
|
||||
yield value.subarray(4);
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
await client.send(new CreateBucketCommand({ Bucket: bucket }));
|
||||
const store = new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket,
|
||||
prefix: 'qinglong/integration',
|
||||
encryption: { mode: 's3' },
|
||||
test(
|
||||
'real S3-compatible service preserves immutable Artifact evidence',
|
||||
{
|
||||
skip:
|
||||
endpoint && accessKeyId && secretAccessKey
|
||||
? false
|
||||
: 'requires QL3_TEST_S3_ENDPOINT and credentials',
|
||||
},
|
||||
async () => {
|
||||
const client = new S3Client({
|
||||
endpoint,
|
||||
region: 'us-east-1',
|
||||
forcePathStyle: true,
|
||||
credentials: { accessKeyId, secretAccessKey },
|
||||
});
|
||||
const stored = await store.put(command, body(content));
|
||||
assert.equal(stored.status, 'stored');
|
||||
assert.equal(
|
||||
stored.sha256,
|
||||
createHash('sha256').update(content).digest('hex'),
|
||||
);
|
||||
const replay = await store.put(command, body(content));
|
||||
assert.equal(replay.status, 'already_stored');
|
||||
await assert.rejects(
|
||||
store.put(command, body(Buffer.from('REAL OBJECT BYTES'))),
|
||||
(error) =>
|
||||
error instanceof S3ClusterRemoteWorkerArtifactStoreError &&
|
||||
error.reason === 'integrity_mismatch',
|
||||
);
|
||||
const objects = await client.send(new ListObjectsV2Command({
|
||||
Bucket: bucket,
|
||||
Prefix: 'qinglong/integration/',
|
||||
}));
|
||||
assert.equal(objects.KeyCount, 1);
|
||||
assert.match(objects.Contents[0].Key, /\/objects\//);
|
||||
} finally {
|
||||
const bucket = `ql3-artifact-${process.pid}-${Date.now()}`.slice(0, 63);
|
||||
const command = Object.freeze({
|
||||
projectId: 'project-s3-integration',
|
||||
runId: 'run-s3-integration',
|
||||
attemptId: 'attempt-s3-integration',
|
||||
logArtifactId: `wlog-${'c'.repeat(30)}`,
|
||||
byteLength: 17,
|
||||
truncated: true,
|
||||
});
|
||||
const content = Buffer.from('real object bytes');
|
||||
const body = (value) =>
|
||||
Object.freeze({
|
||||
async *[Symbol.asyncIterator]() {
|
||||
yield value.subarray(0, 4);
|
||||
yield value.subarray(4);
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const objects = await client.send(new ListObjectsV2Command({
|
||||
Bucket: bucket,
|
||||
}));
|
||||
if (objects.Contents?.length) {
|
||||
await client.send(new DeleteObjectsCommand({
|
||||
await client.send(new CreateBucketCommand({ Bucket: bucket }));
|
||||
const store = new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket,
|
||||
prefix: 'qinglong/integration',
|
||||
encryption: { mode: 's3' },
|
||||
});
|
||||
const stored = await store.put(command, body(content));
|
||||
assert.equal(stored.status, 'stored');
|
||||
assert.equal(
|
||||
stored.sha256,
|
||||
createHash('sha256').update(content).digest('hex'),
|
||||
);
|
||||
const replay = await store.put(command, body(content));
|
||||
assert.equal(replay.status, 'already_stored');
|
||||
const range = await store.readLogRange(
|
||||
{
|
||||
projectId: command.projectId,
|
||||
runId: command.runId,
|
||||
attemptId: command.attemptId,
|
||||
logArtifactId: command.logArtifactId,
|
||||
},
|
||||
{
|
||||
offset: 5,
|
||||
length: 6,
|
||||
},
|
||||
);
|
||||
assert.equal(range.status, 'available');
|
||||
assert.equal(Buffer.from(range.content).toString(), 'object');
|
||||
assert.equal(range.start, 5);
|
||||
assert.equal(range.endExclusive, 11);
|
||||
assert.equal(range.totalBytes, content.byteLength);
|
||||
assert.equal(range.nextOffset, 11);
|
||||
assert.deepEqual(range.truncation, { truncated: true });
|
||||
await assert.rejects(
|
||||
store.put(command, body(Buffer.from('REAL OBJECT BYTES'))),
|
||||
(error) =>
|
||||
error instanceof S3ClusterRemoteWorkerArtifactStoreError &&
|
||||
error.reason === 'integrity_mismatch',
|
||||
);
|
||||
const objects = await client.send(
|
||||
new ListObjectsV2Command({
|
||||
Bucket: bucket,
|
||||
Delete: {
|
||||
Objects: objects.Contents.map(({ Key }) => ({ Key })),
|
||||
Quiet: true,
|
||||
},
|
||||
}));
|
||||
Prefix: 'qinglong/integration/',
|
||||
}),
|
||||
);
|
||||
assert.equal(objects.KeyCount, 1);
|
||||
assert.match(objects.Contents[0].Key, /\/objects\//);
|
||||
} finally {
|
||||
try {
|
||||
const objects = await client.send(
|
||||
new ListObjectsV2Command({
|
||||
Bucket: bucket,
|
||||
}),
|
||||
);
|
||||
if (objects.Contents?.length) {
|
||||
await client.send(
|
||||
new DeleteObjectsCommand({
|
||||
Bucket: bucket,
|
||||
Delete: {
|
||||
Objects: objects.Contents.map(({ Key }) => ({ Key })),
|
||||
Quiet: true,
|
||||
},
|
||||
}),
|
||||
);
|
||||
}
|
||||
await client.send(new DeleteBucketCommand({ Bucket: bucket }));
|
||||
} catch {
|
||||
// Preserve the integration assertion; the ephemeral container is removed.
|
||||
}
|
||||
await client.send(new DeleteBucketCommand({ Bucket: bucket }));
|
||||
} catch {
|
||||
// Preserve the integration assertion; the ephemeral container is removed.
|
||||
client.destroy();
|
||||
}
|
||||
client.destroy();
|
||||
}
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
@@ -6,6 +6,7 @@ const { test } = require('node:test');
|
||||
const {
|
||||
CopyObjectCommand,
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadObjectCommand,
|
||||
PutObjectCommand,
|
||||
} = require('@aws-sdk/client-s3');
|
||||
@@ -57,19 +58,58 @@ class MemoryS3Client {
|
||||
const object = this.objects.get(input.Key);
|
||||
if (!object) throw notFound();
|
||||
const metadata = { ...object.metadata };
|
||||
if (this.options.corruptFinalMetadata && input.Key.includes('/objects/')) {
|
||||
if (
|
||||
this.options.corruptFinalMetadata &&
|
||||
input.Key.includes('/objects/')
|
||||
) {
|
||||
metadata['ql3-content-sha256'] = '0'.repeat(64);
|
||||
}
|
||||
return {
|
||||
ContentLength: object.content.byteLength,
|
||||
ContentType: object.contentType,
|
||||
ChecksumSHA256: this.options.corruptFinalChecksum &&
|
||||
input.Key.includes('/objects/')
|
||||
? Buffer.alloc(32, 9).toString('base64')
|
||||
: checksum(object.content),
|
||||
ETag: `"${checksum(object.content).slice(0, 32)}"`,
|
||||
ChecksumSHA256:
|
||||
this.options.corruptFinalChecksum && input.Key.includes('/objects/')
|
||||
? Buffer.alloc(32, 9).toString('base64')
|
||||
: checksum(object.content),
|
||||
Metadata: metadata,
|
||||
};
|
||||
}
|
||||
if (command instanceof GetObjectCommand) {
|
||||
const object = this.objects.get(input.Key);
|
||||
if (!object || this.options.rangeNotFound) throw notFound();
|
||||
const eTag = `"${checksum(object.content).slice(0, 32)}"`;
|
||||
assert.equal(input.IfMatch, eTag);
|
||||
const match = /^bytes=(\d+)-(\d+)$/.exec(input.Range);
|
||||
assert.ok(match);
|
||||
const start = Number(match[1]);
|
||||
const end = Number(match[2]);
|
||||
let content = object.content.subarray(start, end + 1);
|
||||
if (this.options.shortRangeBody) content = content.subarray(0, -1);
|
||||
if (this.options.oversizedRangeBody) {
|
||||
content = Buffer.concat([content, Buffer.from('x')]);
|
||||
}
|
||||
const metadata = { ...object.metadata };
|
||||
if (this.options.corruptRangeMetadata) {
|
||||
metadata['ql3-run-sha256'] = '0'.repeat(64);
|
||||
}
|
||||
return {
|
||||
ContentLength: end - start + 1,
|
||||
ContentRange: this.options.corruptContentRange
|
||||
? `bytes ${start}-${end}/${object.content.byteLength + 1}`
|
||||
: `bytes ${start}-${end}/${object.content.byteLength}`,
|
||||
ContentType: object.contentType,
|
||||
ETag: this.options.corruptRangeETag ? '"other"' : eTag,
|
||||
Metadata: metadata,
|
||||
Body: {
|
||||
async *[Symbol.asyncIterator]() {
|
||||
const split = Math.min(2, content.byteLength);
|
||||
if (split > 0) yield content.subarray(0, split);
|
||||
if (split < content.byteLength) yield content.subarray(split);
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
if (command instanceof PutObjectCommand) {
|
||||
assert.equal(input.IfNoneMatch, '*');
|
||||
assert.equal(input.ChecksumAlgorithm, 'SHA256');
|
||||
@@ -183,14 +223,25 @@ test('streams to a checksummed temporary object then conditionally promotes it',
|
||||
],
|
||||
);
|
||||
const key = permanentKey(client);
|
||||
assert.match(key, /^tenant-a\/worker-artifacts\/objects\/[a-f0-9]{2}\/[a-f0-9]{64}$/);
|
||||
assert.match(
|
||||
key,
|
||||
/^tenant-a\/worker-artifacts\/objects\/[a-f0-9]{2}\/[a-f0-9]{64}$/,
|
||||
);
|
||||
assert.equal(key.includes(COMMAND.runId), false);
|
||||
assert.equal(client.objects.size, 1);
|
||||
|
||||
const copy = client.commands.find((command) => command instanceof CopyObjectCommand);
|
||||
const copy = client.commands.find(
|
||||
(command) => command instanceof CopyObjectCommand,
|
||||
);
|
||||
assert.equal(copy.input.Metadata['ql3-content-sha256'], CONTENT_SHA256);
|
||||
assert.equal(JSON.stringify(copy.input.Metadata).includes(COMMAND.projectId), false);
|
||||
assert.equal(JSON.stringify(copy.input.Metadata).includes(COMMAND.runId), false);
|
||||
assert.equal(
|
||||
JSON.stringify(copy.input.Metadata).includes(COMMAND.projectId),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
JSON.stringify(copy.input.Metadata).includes(COMMAND.runId),
|
||||
false,
|
||||
);
|
||||
|
||||
const inspected = await adapter.inspect(LOOKUP);
|
||||
assert.deepEqual(inspected, { ...receipt, status: 'already_stored' });
|
||||
@@ -219,6 +270,78 @@ test('exact replay consumes and hashes the whole body without another write', as
|
||||
);
|
||||
});
|
||||
|
||||
test('reads only one ETag-fenced immutable byte range and stable end snapshot', async () => {
|
||||
const client = new MemoryS3Client();
|
||||
const adapter = store(client);
|
||||
await adapter.put(COMMAND, chunks());
|
||||
client.commands.length = 0;
|
||||
|
||||
const result = await adapter.readLogRange(LOOKUP, { offset: 2, length: 4 });
|
||||
assert.equal(result.status, 'available');
|
||||
assert.equal(Buffer.from(result.content).toString(), 'llo ');
|
||||
assert.deepEqual(
|
||||
{
|
||||
start: result.start,
|
||||
endExclusive: result.endExclusive,
|
||||
totalBytes: result.totalBytes,
|
||||
nextOffset: result.nextOffset,
|
||||
truncation: result.truncation,
|
||||
},
|
||||
{
|
||||
start: 2,
|
||||
endExclusive: 6,
|
||||
totalBytes: 11,
|
||||
nextOffset: 6,
|
||||
truncation: { truncated: false },
|
||||
},
|
||||
);
|
||||
assert.deepEqual(
|
||||
client.commands.map((command) => command.constructor.name),
|
||||
['HeadObjectCommand', 'GetObjectCommand'],
|
||||
);
|
||||
assert.equal(client.commands[1].input.Range, 'bytes=2-5');
|
||||
|
||||
client.commands.length = 0;
|
||||
const ended = await adapter.readLogRange(LOOKUP, {
|
||||
offset: 999,
|
||||
length: 4,
|
||||
});
|
||||
assert.equal(ended.status, 'available');
|
||||
assert.equal(ended.content.byteLength, 0);
|
||||
assert.equal(ended.start, 11);
|
||||
assert.equal(ended.totalBytes, 11);
|
||||
assert.deepEqual(
|
||||
client.commands.map((command) => command.constructor.name),
|
||||
['HeadObjectCommand'],
|
||||
);
|
||||
});
|
||||
|
||||
test('maps absent objects and fails closed on range evidence drift', async () => {
|
||||
const absent = new MemoryS3Client();
|
||||
assert.deepEqual(
|
||||
await store(absent).readLogRange(LOOKUP, { offset: 0, length: 1 }),
|
||||
{ status: 'missing' },
|
||||
);
|
||||
for (const option of [
|
||||
'corruptContentRange',
|
||||
'corruptRangeETag',
|
||||
'corruptRangeMetadata',
|
||||
'shortRangeBody',
|
||||
'oversizedRangeBody',
|
||||
]) {
|
||||
const client = new MemoryS3Client();
|
||||
const adapter = store(client);
|
||||
await adapter.put(COMMAND, chunks());
|
||||
client.options[option] = true;
|
||||
await assert.rejects(
|
||||
adapter.readLogRange(LOOKUP, { offset: 0, length: 4 }),
|
||||
(error) =>
|
||||
error instanceof S3ClusterRemoteWorkerArtifactStoreError &&
|
||||
error.reason === 'integrity_mismatch',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('resolves a concurrent conditional-copy winner by immutable inspect', async () => {
|
||||
const client = new MemoryS3Client({ raceOnCopy: true });
|
||||
const receipt = await store(client).put(COMMAND, chunks());
|
||||
@@ -270,38 +393,40 @@ test('temporary cleanup failure is diagnostic and never reverses promotion', asy
|
||||
},
|
||||
}).put(COMMAND, chunks());
|
||||
assert.equal(receipt.status, 'stored');
|
||||
assert.deepEqual(diagnostics, [[
|
||||
'delete unavailable',
|
||||
'temporary_object_cleanup',
|
||||
]]);
|
||||
assert.deepEqual(diagnostics, [
|
||||
['delete unavailable', 'temporary_object_cleanup'],
|
||||
]);
|
||||
assert.equal(client.objects.size, 2);
|
||||
});
|
||||
|
||||
test('requires exact bucket, prefix, encryption and temporary ID configuration', async () => {
|
||||
const client = new MemoryS3Client();
|
||||
assert.throws(
|
||||
() => new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket: 'Invalid_Bucket',
|
||||
encryption: { mode: 's3' },
|
||||
}),
|
||||
() =>
|
||||
new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket: 'Invalid_Bucket',
|
||||
encryption: { mode: 's3' },
|
||||
}),
|
||||
/bucket is invalid/,
|
||||
);
|
||||
assert.throws(
|
||||
() => new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket: 'valid-bucket',
|
||||
prefix: '../escape',
|
||||
encryption: { mode: 's3' },
|
||||
}),
|
||||
() =>
|
||||
new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket: 'valid-bucket',
|
||||
prefix: '../escape',
|
||||
encryption: { mode: 's3' },
|
||||
}),
|
||||
/prefix is invalid/,
|
||||
);
|
||||
assert.throws(
|
||||
() => new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket: 'valid-bucket',
|
||||
encryption: { mode: 'kms' },
|
||||
}),
|
||||
() =>
|
||||
new S3ClusterRemoteWorkerArtifactStore({
|
||||
client,
|
||||
bucket: 'valid-bucket',
|
||||
encryption: { mode: 'kms' },
|
||||
}),
|
||||
/encryption is invalid/,
|
||||
);
|
||||
await assert.rejects(
|
||||
@@ -355,8 +480,7 @@ test('propagates KMS and expected-owner fences to both sides of promotion', asyn
|
||||
|
||||
test('never deletes a colliding temporary object it cannot prove it owns', async () => {
|
||||
const client = new MemoryS3Client();
|
||||
const temporaryKey =
|
||||
`tenant-a/worker-artifacts/temporary/${TEMPORARY_ID}`;
|
||||
const temporaryKey = `tenant-a/worker-artifacts/temporary/${TEMPORARY_ID}`;
|
||||
client.objects.set(temporaryKey, {
|
||||
content: Buffer.from('other operation'),
|
||||
contentType: 'application/octet-stream',
|
||||
|
||||
@@ -22,6 +22,7 @@ import type { LocalApiRunListRoute } from '../run/runListRoute';
|
||||
import type { LocalApiRunReadRoute } from '../run/runReadRoute';
|
||||
import type { LocalApiRunStepListRoute } from '../run/runStepListRoute';
|
||||
import type { LocalApiRunCancellationRoute } from '../run/runCancellationRoute';
|
||||
import type { LocalApiRunAttemptLogReadRoute } from '../run/runAttemptLogReadRoute';
|
||||
import type { LocalApiTaskListRoute } from '../task/taskListRoute';
|
||||
import type { LocalApiTaskReadRoute } from '../task/taskReadRoute';
|
||||
import type { LocalApiTaskStartRoute } from '../task/taskStartRoute';
|
||||
@@ -50,6 +51,14 @@ export type LocalApiAdmissionOperation =
|
||||
runId: string;
|
||||
input: Readonly<BoundedRunStepListInput>;
|
||||
}>
|
||||
| Readonly<{
|
||||
operationId: 'run.log.read';
|
||||
projectId: string;
|
||||
runId: string;
|
||||
attemptId: string;
|
||||
offset: number;
|
||||
length: number;
|
||||
}>
|
||||
| Readonly<{
|
||||
operationId: 'run.cancel';
|
||||
projectId: string;
|
||||
@@ -99,6 +108,7 @@ export interface LocalApiAdmissionOptions {
|
||||
readonly runEventListRoute: LocalApiRunEventListRoute;
|
||||
readonly runStepListRoute: LocalApiRunStepListRoute;
|
||||
readonly runCancellationRoute: LocalApiRunCancellationRoute;
|
||||
readonly runAttemptLogReadRoute: LocalApiRunAttemptLogReadRoute;
|
||||
readonly taskListRoute: LocalApiTaskListRoute;
|
||||
readonly taskReadRoute: LocalApiTaskReadRoute;
|
||||
readonly taskStartRoute: LocalApiTaskStartRoute;
|
||||
@@ -176,6 +186,7 @@ export function createLocalApiAdmission(
|
||||
typeof options.runEventListRoute?.handle !== 'function' ||
|
||||
typeof options.runStepListRoute?.handle !== 'function' ||
|
||||
typeof options.runCancellationRoute?.handle !== 'function' ||
|
||||
typeof options.runAttemptLogReadRoute?.handle !== 'function' ||
|
||||
typeof options.taskListRoute?.handle !== 'function' ||
|
||||
typeof options.taskReadRoute?.handle !== 'function' ||
|
||||
typeof options.taskStartRoute?.handle !== 'function' ||
|
||||
@@ -236,12 +247,14 @@ export function createLocalApiAdmission(
|
||||
request.operation.projectId,
|
||||
request.operation.operationId === 'run.cancel'
|
||||
? 'run.stop'
|
||||
: request.operation.operationId === 'run.log.read'
|
||||
? 'artifact.read'
|
||||
: request.operation.operationId === 'task.start'
|
||||
? 'run.start'
|
||||
? 'run.start'
|
||||
: request.operation.operationId === 'task.list' ||
|
||||
request.operation.operationId === 'task.get'
|
||||
? 'task.read'
|
||||
: 'run.read',
|
||||
request.operation.operationId === 'task.get'
|
||||
? 'task.read'
|
||||
: 'run.read',
|
||||
),
|
||||
);
|
||||
} catch {
|
||||
@@ -279,9 +292,15 @@ export function createLocalApiAdmission(
|
||||
),
|
||||
);
|
||||
if (auditFailure) return auditFailure;
|
||||
if (decision.effect === 'deny') return response(403, 'forbidden');
|
||||
if (decision.effect === 'deny') {
|
||||
return request.operation.operationId === 'run.log.read'
|
||||
? response(404, 'artifact_not_found')
|
||||
: response(403, 'forbidden');
|
||||
}
|
||||
if (decision.effect === 'require_approval') {
|
||||
return response(403, 'approval_required');
|
||||
return request.operation.operationId === 'run.log.read'
|
||||
? response(404, 'artifact_not_found')
|
||||
: response(403, 'approval_required');
|
||||
}
|
||||
if (request.signal.aborted) return response(503, 'request_unavailable');
|
||||
try {
|
||||
@@ -337,6 +356,16 @@ export function createLocalApiAdmission(
|
||||
principal: authenticated.principal,
|
||||
policyFence: decision.fence,
|
||||
});
|
||||
case 'run.log.read':
|
||||
if (body !== null) return response(400, 'invalid_request_body');
|
||||
return options.runAttemptLogReadRoute.handle({
|
||||
projectId: request.operation.projectId,
|
||||
runId: request.operation.runId,
|
||||
attemptId: request.operation.attemptId,
|
||||
offset: request.operation.offset,
|
||||
length: request.operation.length,
|
||||
signal: request.signal,
|
||||
});
|
||||
case 'task.list':
|
||||
if (body !== null) return response(400, 'invalid_request_body');
|
||||
return options.taskListRoute.handle({
|
||||
|
||||
@@ -15,6 +15,7 @@ import { createLocalApiRunReadRoute } from '../run/runReadRoute';
|
||||
import { createLocalApiRunEventListRoute } from '../run/runEventListRoute';
|
||||
import { createLocalApiRunStepListRoute } from '../run/runStepListRoute';
|
||||
import { createLocalApiRunCancellationRoute } from '../run/runCancellationRoute';
|
||||
import { createLocalApiRunAttemptLogReadRoute } from '../run/runAttemptLogReadRoute';
|
||||
import { createLocalApiTaskListRoute } from '../task/taskListRoute';
|
||||
import { createLocalApiTaskReadRoute } from '../task/taskReadRoute';
|
||||
import { createLocalApiTaskStartRoute } from '../task/taskStartRoute';
|
||||
@@ -109,6 +110,9 @@ export function createLocalApiProductSurface(
|
||||
authority.runCancellation,
|
||||
options.randomUuid ?? randomUUID,
|
||||
);
|
||||
const runAttemptLogReadRoute = createLocalApiRunAttemptLogReadRoute(
|
||||
authority.runAttemptLogRead,
|
||||
);
|
||||
const taskListRoute = createLocalApiTaskListRoute(
|
||||
authority.taskDefinitions,
|
||||
);
|
||||
@@ -128,6 +132,7 @@ export function createLocalApiProductSurface(
|
||||
runEventListRoute,
|
||||
runStepListRoute,
|
||||
runCancellationRoute,
|
||||
runAttemptLogReadRoute,
|
||||
taskListRoute,
|
||||
taskReadRoute,
|
||||
taskStartRoute,
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
import {
|
||||
InvalidRunAttemptLogReadError,
|
||||
RunAttemptLogReadUnavailableError,
|
||||
type RunAttemptLogReadRequest,
|
||||
type RunAttemptLogReadResult,
|
||||
} from '@qinglong/runtime-core/run-attempt-log-read';
|
||||
|
||||
import type { LocalApiResponse } from '../transport/contract';
|
||||
|
||||
export interface LocalApiRunAttemptLogReadCapability {
|
||||
read(
|
||||
request: Readonly<RunAttemptLogReadRequest>,
|
||||
): Promise<RunAttemptLogReadResult>;
|
||||
}
|
||||
|
||||
export interface LocalApiRunAttemptLogReadRequest {
|
||||
readonly projectId: string;
|
||||
readonly runId: string;
|
||||
readonly attemptId: string;
|
||||
readonly offset: number;
|
||||
readonly length: number;
|
||||
readonly signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export interface LocalApiRunAttemptLogReadRoute {
|
||||
handle(
|
||||
request: Readonly<LocalApiRunAttemptLogReadRequest>,
|
||||
): Promise<LocalApiResponse>;
|
||||
}
|
||||
|
||||
function response(
|
||||
statusCode: number,
|
||||
body: Readonly<Record<string, unknown>>,
|
||||
): LocalApiResponse {
|
||||
return Object.freeze({ statusCode, body: Object.freeze(body) });
|
||||
}
|
||||
|
||||
function projection(
|
||||
result: Extract<RunAttemptLogReadResult, { readonly status: 'available' }>,
|
||||
): Readonly<Record<string, unknown>> {
|
||||
return Object.freeze({
|
||||
schema: 'qinglong/run-attempt-log-read-result@v1',
|
||||
status: 'available',
|
||||
projectId: result.projectId,
|
||||
runId: result.runId,
|
||||
attemptId: result.attemptId,
|
||||
range: Object.freeze({
|
||||
start: result.start,
|
||||
endExclusive: result.endExclusive,
|
||||
totalBytes: result.totalBytes,
|
||||
...(result.nextOffset === undefined
|
||||
? {}
|
||||
: { nextOffset: result.nextOffset }),
|
||||
}),
|
||||
encoding: 'base64',
|
||||
content: Buffer.from(
|
||||
result.content.buffer,
|
||||
result.content.byteOffset,
|
||||
result.content.byteLength,
|
||||
).toString('base64'),
|
||||
truncation: result.truncation,
|
||||
});
|
||||
}
|
||||
|
||||
export function createLocalApiRunAttemptLogReadRoute(
|
||||
capability: LocalApiRunAttemptLogReadCapability,
|
||||
): Readonly<LocalApiRunAttemptLogReadRoute> {
|
||||
if (!capability || typeof capability.read !== 'function') {
|
||||
throw new TypeError('Local API Run Attempt log read capability is invalid');
|
||||
}
|
||||
return Object.freeze({
|
||||
async handle(request: Readonly<LocalApiRunAttemptLogReadRequest>) {
|
||||
try {
|
||||
const result = await capability.read({
|
||||
projectId: request.projectId,
|
||||
runId: request.runId,
|
||||
attemptId: request.attemptId,
|
||||
range: Object.freeze({
|
||||
offset: request.offset,
|
||||
length: request.length,
|
||||
}),
|
||||
...(request.signal === undefined ? {} : { signal: request.signal }),
|
||||
});
|
||||
if (result.status === 'not_found') {
|
||||
return response(404, { code: 'artifact_not_found' });
|
||||
}
|
||||
if (result.status === 'pending') {
|
||||
return response(202, {
|
||||
schema: 'qinglong/run-attempt-log-read-result@v1',
|
||||
status: 'pending',
|
||||
projectId: result.projectId,
|
||||
runId: result.runId,
|
||||
attemptId: result.attemptId,
|
||||
});
|
||||
}
|
||||
if (result.status === 'missing') {
|
||||
return response(503, { code: 'artifact_unavailable' });
|
||||
}
|
||||
return response(200, projection(result));
|
||||
} catch (error) {
|
||||
if (error instanceof InvalidRunAttemptLogReadError) {
|
||||
return response(400, { code: 'invalid_run_log_read_request' });
|
||||
}
|
||||
if (error instanceof RunAttemptLogReadUnavailableError) {
|
||||
return response(503, { code: 'artifact_unavailable' });
|
||||
}
|
||||
return response(503, { code: 'artifact_unavailable' });
|
||||
}
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -28,6 +28,8 @@ const RUN_STEP_LIST_ROUTE_PATTERN =
|
||||
/^\/api\/v3\/projects\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/runs\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/steps$/;
|
||||
const RUN_CANCELLATION_ROUTE_PATTERN =
|
||||
/^\/api\/v3\/projects\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/runs\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/cancellation$/;
|
||||
const RUN_ATTEMPT_LOG_READ_ROUTE_PATTERN =
|
||||
/^\/api\/v3\/projects\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/runs\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/attempts\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/log$/;
|
||||
const TASK_LIST_ROUTE_PATTERN =
|
||||
/^\/api\/v3\/projects\/([A-Za-z0-9][A-Za-z0-9._:-]{0,127})\/tasks$/;
|
||||
const TASK_READ_ROUTE_PATTERN =
|
||||
@@ -44,6 +46,7 @@ type LocalApiRouteResolution =
|
||||
| 'invalid_run_list_query'
|
||||
| 'invalid_run_event_list_query'
|
||||
| 'invalid_run_step_list_query'
|
||||
| 'invalid_run_log_read_query'
|
||||
| 'invalid_task_list_query';
|
||||
}>;
|
||||
|
||||
@@ -340,10 +343,7 @@ function parseTaskListQuery(
|
||||
}
|
||||
const name = field.slice(0, separator);
|
||||
const value = field.slice(separator + 1);
|
||||
if (
|
||||
values.has(name) ||
|
||||
(name !== 'limit' && name !== 'after_task_id')
|
||||
) {
|
||||
if (values.has(name) || (name !== 'limit' && name !== 'after_task_id')) {
|
||||
throw new TypeError();
|
||||
}
|
||||
values.set(name, value);
|
||||
@@ -363,13 +363,58 @@ function parseTaskListQuery(
|
||||
}
|
||||
return Object.freeze({
|
||||
...(limit === undefined ? {} : { limit }),
|
||||
...(taskId === undefined
|
||||
? {}
|
||||
: { after: Object.freeze({ taskId }) }),
|
||||
...(taskId === undefined ? {} : { after: Object.freeze({ taskId }) }),
|
||||
});
|
||||
}
|
||||
|
||||
function route(request: IncomingMessage): LocalApiRouteResolution | null {
|
||||
function parseRunAttemptLogReadQuery(
|
||||
rawQuery: string | undefined,
|
||||
profile: LocalApplicationProfile,
|
||||
): Readonly<{ offset: number; length: number }> {
|
||||
const defaultLength = profile === 'edge' ? 16 * 1024 : 32 * 1024;
|
||||
if (rawQuery === undefined) {
|
||||
return Object.freeze({ offset: 0, length: defaultLength });
|
||||
}
|
||||
if (rawQuery.length === 0) throw new TypeError();
|
||||
const values = new Map<string, string>();
|
||||
for (const field of rawQuery.split('&')) {
|
||||
const separator = field.indexOf('=');
|
||||
if (
|
||||
separator < 1 ||
|
||||
separator !== field.lastIndexOf('=') ||
|
||||
separator === field.length - 1
|
||||
) {
|
||||
throw new TypeError();
|
||||
}
|
||||
const name = field.slice(0, separator);
|
||||
const value = field.slice(separator + 1);
|
||||
if (values.has(name) || (name !== 'offset' && name !== 'length')) {
|
||||
throw new TypeError();
|
||||
}
|
||||
values.set(name, value);
|
||||
}
|
||||
const rawOffset = values.get('offset');
|
||||
const offset = rawOffset === undefined ? 0 : Number(rawOffset);
|
||||
const rawLength = values.get('length');
|
||||
const length = rawLength === undefined ? defaultLength : Number(rawLength);
|
||||
if (
|
||||
!Number.isSafeInteger(offset) ||
|
||||
offset < 0 ||
|
||||
(rawOffset !== undefined && String(offset) !== rawOffset) ||
|
||||
!Number.isSafeInteger(length) ||
|
||||
length < 1 ||
|
||||
length > 32 * 1024 ||
|
||||
(rawLength !== undefined && String(length) !== rawLength)
|
||||
) {
|
||||
throw new TypeError();
|
||||
}
|
||||
return Object.freeze({ offset, length });
|
||||
}
|
||||
|
||||
function route(
|
||||
request: IncomingMessage,
|
||||
profile: LocalApplicationProfile,
|
||||
): LocalApiRouteResolution | null {
|
||||
const rawUrl = request.url;
|
||||
if (
|
||||
typeof rawUrl !== 'string' ||
|
||||
@@ -403,6 +448,20 @@ function route(request: IncomingMessage): LocalApiRouteResolution | null {
|
||||
: null;
|
||||
}
|
||||
if (request.method !== 'GET') return null;
|
||||
const runAttemptLogReadMatch = RUN_ATTEMPT_LOG_READ_ROUTE_PATTERN.exec(path);
|
||||
if (runAttemptLogReadMatch) {
|
||||
try {
|
||||
return Object.freeze({
|
||||
operationId: 'run.log.read',
|
||||
projectId: runAttemptLogReadMatch[1]!,
|
||||
runId: runAttemptLogReadMatch[2]!,
|
||||
attemptId: runAttemptLogReadMatch[3]!,
|
||||
...parseRunAttemptLogReadQuery(rawQuery, profile),
|
||||
});
|
||||
} catch {
|
||||
return Object.freeze({ errorCode: 'invalid_run_log_read_query' });
|
||||
}
|
||||
}
|
||||
const taskReadMatch = TASK_READ_ROUTE_PATTERN.exec(path);
|
||||
if (taskReadMatch) {
|
||||
return rawQuery === undefined
|
||||
@@ -552,7 +611,7 @@ export async function startLocalApiHttpSurface(
|
||||
send(response, requestId, errorResponse(503, 'server_overloaded'));
|
||||
return;
|
||||
}
|
||||
const resolvedRoute = route(request);
|
||||
const resolvedRoute = route(request, options.profile);
|
||||
if (!resolvedRoute) {
|
||||
send(response, requestId, errorResponse(404, 'route_not_found'));
|
||||
return;
|
||||
@@ -608,9 +667,9 @@ export async function startLocalApiHttpSurface(
|
||||
error instanceof RangeError
|
||||
? 'request_body_too_large'
|
||||
: error instanceof Error &&
|
||||
error.message === 'request_unavailable'
|
||||
? 'request_unavailable'
|
||||
: 'invalid_request_body';
|
||||
error.message === 'request_unavailable'
|
||||
? 'request_unavailable'
|
||||
: 'invalid_request_body';
|
||||
send(
|
||||
response,
|
||||
requestId,
|
||||
@@ -618,8 +677,8 @@ export async function startLocalApiHttpSurface(
|
||||
code === 'request_body_too_large'
|
||||
? 413
|
||||
: code === 'request_unavailable'
|
||||
? 503
|
||||
: 400,
|
||||
? 503
|
||||
: 400,
|
||||
code,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -101,6 +101,14 @@ function fixture(overrides = {}) {
|
||||
return { statusCode: 202, body: { status: 'accepted' } };
|
||||
},
|
||||
},
|
||||
runAttemptLogReadRoute: {
|
||||
async handle(value) {
|
||||
events.push(
|
||||
`log:${value.projectId}:${value.runId}:${value.attemptId}:${value.offset}:${value.length}`,
|
||||
);
|
||||
return { statusCode: 200, body: { status: 'available' } };
|
||||
},
|
||||
},
|
||||
taskListRoute: {
|
||||
async handle(value) {
|
||||
events.push(`tasks:${value.projectId}:${value.input.limit ?? 32}`);
|
||||
@@ -148,6 +156,51 @@ test('authenticates, authorizes, durably audits and re-confirms before reading',
|
||||
]);
|
||||
});
|
||||
|
||||
test('uses artifact.read and masks denied or approval-fenced log existence', async () => {
|
||||
const operation = Object.freeze({
|
||||
operationId: 'run.log.read',
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
offset: 4,
|
||||
length: 16,
|
||||
});
|
||||
const allowed = fixture();
|
||||
assert.deepEqual(await execute(allowed.admission, request({ operation })), {
|
||||
statusCode: 200,
|
||||
body: { status: 'available' },
|
||||
});
|
||||
assert.deepEqual(allowed.events, [
|
||||
'authenticate',
|
||||
'authorize:artifact.read:prj_default',
|
||||
'audit:allowed:run.log.read',
|
||||
'confirm',
|
||||
'log:prj_default:run_123:attempt_123:4:16',
|
||||
]);
|
||||
|
||||
for (const effect of ['deny', 'require_approval']) {
|
||||
let routed = false;
|
||||
const denied = fixture({
|
||||
policy: {
|
||||
async authorize() {
|
||||
return { effect, reasons: ['masked'], fence: null };
|
||||
},
|
||||
},
|
||||
runAttemptLogReadRoute: {
|
||||
async handle() {
|
||||
routed = true;
|
||||
throw new Error('must not route');
|
||||
},
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await execute(denied.admission, request({ operation })), {
|
||||
statusCode: 404,
|
||||
body: { code: 'artifact_not_found' },
|
||||
});
|
||||
assert.equal(routed, false);
|
||||
}
|
||||
});
|
||||
|
||||
test('uses the same admission chain with a route-owned run.list audit identity', async () => {
|
||||
const { admission, events } = fixture();
|
||||
assert.deepEqual(
|
||||
|
||||
@@ -74,60 +74,71 @@ test('serves only the fixed canonical loopback Run route and drains idempotently
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
admission: preparedAdmission(async (value, body) => {
|
||||
observed.push(value);
|
||||
if (
|
||||
value.operation.operationId === 'run.cancel' ||
|
||||
value.operation.operationId === 'task.start'
|
||||
) {
|
||||
return { statusCode: 202, body: { accepted: body } };
|
||||
}
|
||||
if (value.operation.operationId === 'run.get') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: { run: { id: value.operation.runId } },
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'run.events.list') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
events: [],
|
||||
hasMore: false,
|
||||
nextAfterSequence: value.operation.input.afterSequence ?? 0,
|
||||
},
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'run.steps.list') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
steps: [],
|
||||
hasMore: false,
|
||||
next: value.operation.input.after ?? null,
|
||||
},
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'task.list') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
tasks: [],
|
||||
hasMore: false,
|
||||
input: value.operation.input,
|
||||
},
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'task.get') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: { task: { taskId: value.operation.taskId } },
|
||||
};
|
||||
}
|
||||
observed.push(value);
|
||||
if (
|
||||
value.operation.operationId === 'run.cancel' ||
|
||||
value.operation.operationId === 'task.start'
|
||||
) {
|
||||
return { statusCode: 202, body: { accepted: body } };
|
||||
}
|
||||
if (value.operation.operationId === 'run.get') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: { runs: [], hasMore: false, input: value.operation.input },
|
||||
body: { run: { id: value.operation.runId } },
|
||||
};
|
||||
}),
|
||||
}
|
||||
if (value.operation.operationId === 'run.events.list') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
events: [],
|
||||
hasMore: false,
|
||||
nextAfterSequence: value.operation.input.afterSequence ?? 0,
|
||||
},
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'run.steps.list') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
steps: [],
|
||||
hasMore: false,
|
||||
next: value.operation.input.after ?? null,
|
||||
},
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'run.log.read') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
range: {
|
||||
offset: value.operation.offset,
|
||||
length: value.operation.length,
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'task.list') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
tasks: [],
|
||||
hasMore: false,
|
||||
input: value.operation.input,
|
||||
},
|
||||
};
|
||||
}
|
||||
if (value.operation.operationId === 'task.get') {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: { task: { taskId: value.operation.taskId } },
|
||||
};
|
||||
}
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: { runs: [], hasMore: false, input: value.operation.input },
|
||||
};
|
||||
}),
|
||||
randomUuid: () => '019f70c0-0000-4000-8000-000000000003',
|
||||
});
|
||||
t.after(() => surface.stopAndDrain());
|
||||
@@ -212,10 +223,7 @@ test('serves only the fixed canonical loopback Run route and drains idempotently
|
||||
input: { after: { taskId: 'task_100' }, limit: 8 },
|
||||
});
|
||||
|
||||
const task = await request(
|
||||
port,
|
||||
'/api/v3/projects/prj_default/tasks/task_1',
|
||||
);
|
||||
const task = await request(port, '/api/v3/projects/prj_default/tasks/task_1');
|
||||
assert.deepEqual(task.body, { task: { taskId: 'task_1' } });
|
||||
assert.deepEqual(observed[5].operation, {
|
||||
operationId: 'task.get',
|
||||
@@ -275,6 +283,28 @@ test('serves only the fixed canonical loopback Run route and drains idempotently
|
||||
taskId: 'task_1',
|
||||
});
|
||||
|
||||
const log = await request(
|
||||
port,
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log?offset=4&length=32',
|
||||
);
|
||||
assert.deepEqual(log.body, { range: { offset: 4, length: 32 } });
|
||||
assert.deepEqual(observed[8].operation, {
|
||||
operationId: 'run.log.read',
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_1',
|
||||
offset: 4,
|
||||
length: 32,
|
||||
});
|
||||
|
||||
const defaultLog = await request(
|
||||
port,
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log',
|
||||
);
|
||||
assert.deepEqual(defaultLog.body, {
|
||||
range: { offset: 0, length: 16 * 1024 },
|
||||
});
|
||||
|
||||
for (const invalidPath of [
|
||||
'/api/v3/projects/prj_default/runs/run_123?expanded=true',
|
||||
'/api/v3/projects/prj_default/runs/run%5f123',
|
||||
@@ -299,6 +329,18 @@ test('serves only the fixed canonical loopback Run route and drains idempotently
|
||||
assert.equal(invalid.statusCode, 400);
|
||||
assert.deepEqual(invalid.body, { code: 'invalid_run_list_query' });
|
||||
}
|
||||
for (const invalidQuery of [
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log?',
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log?offset=-1',
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log?offset=04',
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log?length=0',
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log?length=32769',
|
||||
'/api/v3/projects/prj_default/runs/run_123/attempts/attempt_1/log?unknown=1',
|
||||
]) {
|
||||
const invalid = await request(port, invalidQuery);
|
||||
assert.equal(invalid.statusCode, 400);
|
||||
assert.deepEqual(invalid.body, { code: 'invalid_run_log_read_query' });
|
||||
}
|
||||
for (const invalidQuery of [
|
||||
'/api/v3/projects/prj_default/tasks?',
|
||||
'/api/v3/projects/prj_default/tasks?limit=08',
|
||||
@@ -332,7 +374,7 @@ test('serves only the fixed canonical loopback Run route and drains idempotently
|
||||
assert.equal(invalid.statusCode, 400);
|
||||
assert.deepEqual(invalid.body, { code: 'invalid_run_step_list_query' });
|
||||
}
|
||||
assert.equal(observed.length, 8);
|
||||
assert.equal(observed.length, 10);
|
||||
assert.deepEqual(
|
||||
await Promise.all([surface.stopAndDrain(), surface.stopAndDrain()]),
|
||||
['stopped', 'stopped'],
|
||||
@@ -347,9 +389,9 @@ test('rejects GET bodies without invoking the prepared route handler', async (t)
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
admission: preparedAdmission(async () => {
|
||||
handlers += 1;
|
||||
return { statusCode: 200, body: {} };
|
||||
}),
|
||||
handlers += 1;
|
||||
return { statusCode: 200, body: {} };
|
||||
}),
|
||||
});
|
||||
t.after(() => surface.stopAndDrain());
|
||||
const response = await request(
|
||||
@@ -477,15 +519,15 @@ test('serves the reviewed worst-case 64-item Run list inside the fixed response
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
admission: preparedAdmission(async () => {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
runs: Object.freeze(Array.from({ length: 64 }, () => item)),
|
||||
hasMore: true,
|
||||
next: { createdAtMs: Number.MAX_SAFE_INTEGER, runId: id128 },
|
||||
},
|
||||
};
|
||||
}),
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
runs: Object.freeze(Array.from({ length: 64 }, () => item)),
|
||||
hasMore: true,
|
||||
next: { createdAtMs: Number.MAX_SAFE_INTEGER, runId: id128 },
|
||||
},
|
||||
};
|
||||
}),
|
||||
});
|
||||
t.after(() => surface.stopAndDrain());
|
||||
const response = await request(
|
||||
@@ -545,15 +587,15 @@ test('serves the reviewed worst-case 64-item RunEvent list inside the fixed resp
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
admission: preparedAdmission(async () => {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
events: Object.freeze(Array.from({ length: 64 }, () => event)),
|
||||
hasMore: true,
|
||||
nextAfterSequence: event.sequence,
|
||||
},
|
||||
};
|
||||
}),
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
events: Object.freeze(Array.from({ length: 64 }, () => event)),
|
||||
hasMore: true,
|
||||
nextAfterSequence: event.sequence,
|
||||
},
|
||||
};
|
||||
}),
|
||||
});
|
||||
t.after(() => surface.stopAndDrain());
|
||||
const response = await request(
|
||||
@@ -589,15 +631,15 @@ test('serves the reviewed worst-case 64-item Run Step list inside the fixed resp
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
admission: preparedAdmission(async () => {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
steps: Object.freeze(Array.from({ length: 64 }, () => item)),
|
||||
hasMore: true,
|
||||
next: { stepKey: id128, stepRunId: id128 },
|
||||
},
|
||||
};
|
||||
}),
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
steps: Object.freeze(Array.from({ length: 64 }, () => item)),
|
||||
hasMore: true,
|
||||
next: { stepKey: id128, stepRunId: id128 },
|
||||
},
|
||||
};
|
||||
}),
|
||||
});
|
||||
t.after(() => surface.stopAndDrain());
|
||||
const response = await request(
|
||||
@@ -621,13 +663,13 @@ test('bounds Edge admission concurrency and drains accepted work', async (t) =>
|
||||
host: '127.0.0.1',
|
||||
port,
|
||||
admission: preparedAdmission(async (value) => {
|
||||
admissions += 1;
|
||||
await barrier;
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: { run: { id: value.operation.runId } },
|
||||
};
|
||||
}),
|
||||
admissions += 1;
|
||||
await barrier;
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: { run: { id: value.operation.runId } },
|
||||
};
|
||||
}),
|
||||
});
|
||||
t.after(() => surface.stopAndDrain());
|
||||
const accepted = Array.from({ length: 4 }, () =>
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const { test } = require('node:test');
|
||||
|
||||
const {
|
||||
RunAttemptLogReadUnavailableError,
|
||||
} = require('@qinglong/runtime-core/run-attempt-log-read');
|
||||
const {
|
||||
createLocalApiRunAttemptLogReadRoute,
|
||||
} = require('../dist/run/runAttemptLogReadRoute.js');
|
||||
|
||||
function request(overrides = {}) {
|
||||
return {
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
offset: 2,
|
||||
length: 16,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test('projects an available byte range as bounded base64 JSON', async () => {
|
||||
const route = createLocalApiRunAttemptLogReadRoute({
|
||||
async read(value) {
|
||||
assert.deepEqual(value.range, { offset: 2, length: 16 });
|
||||
return {
|
||||
status: 'available',
|
||||
projectId: value.projectId,
|
||||
runId: value.runId,
|
||||
attemptId: value.attemptId,
|
||||
logArtifactId: `local-${'a'.repeat(30)}`,
|
||||
content: Buffer.from('hello'),
|
||||
start: 2,
|
||||
endExclusive: 7,
|
||||
totalBytes: 9,
|
||||
nextOffset: 7,
|
||||
truncation: { truncated: 'unknown' },
|
||||
};
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await route.handle(request()), {
|
||||
statusCode: 200,
|
||||
body: {
|
||||
schema: 'qinglong/run-attempt-log-read-result@v1',
|
||||
status: 'available',
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
range: { start: 2, endExclusive: 7, totalBytes: 9, nextOffset: 7 },
|
||||
encoding: 'base64',
|
||||
content: Buffer.from('hello').toString('base64'),
|
||||
truncation: { truncated: 'unknown' },
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test('maps pending, masked absence, missing storage and unavailable evidence', async () => {
|
||||
const cases = [
|
||||
[
|
||||
{
|
||||
status: 'pending',
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
},
|
||||
{
|
||||
statusCode: 202,
|
||||
body: {
|
||||
schema: 'qinglong/run-attempt-log-read-result@v1',
|
||||
status: 'pending',
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
},
|
||||
},
|
||||
],
|
||||
[
|
||||
{ status: 'not_found' },
|
||||
{ statusCode: 404, body: { code: 'artifact_not_found' } },
|
||||
],
|
||||
[
|
||||
{
|
||||
status: 'missing',
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
logArtifactId: `local-${'a'.repeat(30)}`,
|
||||
},
|
||||
{ statusCode: 503, body: { code: 'artifact_unavailable' } },
|
||||
],
|
||||
];
|
||||
for (const [result, expected] of cases) {
|
||||
const route = createLocalApiRunAttemptLogReadRoute({
|
||||
async read() {
|
||||
return result;
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await route.handle(request()), expected);
|
||||
}
|
||||
const unavailable = createLocalApiRunAttemptLogReadRoute({
|
||||
async read() {
|
||||
throw new RunAttemptLogReadUnavailableError();
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await unavailable.handle(request()), {
|
||||
statusCode: 503,
|
||||
body: { code: 'artifact_unavailable' },
|
||||
});
|
||||
});
|
||||
@@ -24,6 +24,9 @@ const {
|
||||
const {
|
||||
compileLocalCommandTaskDefinition,
|
||||
} = require('@qinglong/runtime-core/task-definition-execution-compiler');
|
||||
const {
|
||||
RunAttemptLogReadService,
|
||||
} = require('@qinglong/runtime-core/run-attempt-log-read');
|
||||
const {
|
||||
createBuiltInTaskSpecSemanticRegistry,
|
||||
} = require('@qinglong/runtime-core/task-spec-semantic');
|
||||
@@ -34,11 +37,16 @@ const {
|
||||
const {
|
||||
createLocalApiProductSurface,
|
||||
} = require('../dist/application-runtime/localApiProductSurface.js');
|
||||
const {
|
||||
LocalRunAttemptLogRangeReader,
|
||||
} = require('../../ql3-local-execution/dist/artifact-read/localRunAttemptLogRangeReader.js');
|
||||
|
||||
const NOW = 1_800_000_000_000;
|
||||
const PEPPER_KEY_ID = 'local-api-pepper-v1';
|
||||
const CREDENTIAL_ID = 'local-api-owner';
|
||||
const RUN_ID = 'run_local_api_1';
|
||||
const ATTEMPT_ID = 'attempt_local_api_1';
|
||||
const LOG_ARTIFACT_ID = `local-${'a'.repeat(30)}`;
|
||||
const SECRET = Buffer.alloc(32, 81).toString('base64url');
|
||||
const PEPPER = Buffer.alloc(32, 82).toString('base64url');
|
||||
const TOKEN = formatApiCredentialToken(CREDENTIAL_ID, SECRET);
|
||||
@@ -206,15 +214,18 @@ function seed(databasePath, materialDigest) {
|
||||
enabled: true,
|
||||
occurredAtMs: NOW - 200,
|
||||
};
|
||||
const taskDefinition = createTaskDefinitionRecord({
|
||||
...taskCommand,
|
||||
spec: taskSemantics.normalize({
|
||||
projectId: taskCommand.projectId,
|
||||
taskId: taskCommand.taskId,
|
||||
kind: taskCommand.kind,
|
||||
spec: taskCommand.spec,
|
||||
}),
|
||||
}, NOW - 200);
|
||||
const taskDefinition = createTaskDefinitionRecord(
|
||||
{
|
||||
...taskCommand,
|
||||
spec: taskSemantics.normalize({
|
||||
projectId: taskCommand.projectId,
|
||||
taskId: taskCommand.taskId,
|
||||
kind: taskCommand.kind,
|
||||
spec: taskCommand.spec,
|
||||
}),
|
||||
},
|
||||
NOW - 200,
|
||||
);
|
||||
const taskExecution = compileLocalCommandTaskDefinition(
|
||||
taskDefinition,
|
||||
taskSemantics,
|
||||
@@ -298,6 +309,15 @@ function seed(databasePath, materialDigest) {
|
||||
'manual', 'runtime', 'running', 1, 1, 0, ?)`,
|
||||
)
|
||||
.run(RUN_ID, NOW - 100);
|
||||
client
|
||||
.prepare(
|
||||
`INSERT INTO "RunAttempts" (
|
||||
"id", "run_id", "attempt", "status", "executor_type",
|
||||
"log_artifact_id", "callback_sequence", "created_at_ms",
|
||||
"started_at_ms"
|
||||
) VALUES (?, ?, 1, 'running', 'local_process', ?, 0, ?, ?)`,
|
||||
)
|
||||
.run(ATTEMPT_ID, RUN_ID, LOG_ARTIFACT_ID, NOW - 90, NOW - 80);
|
||||
client
|
||||
.prepare(
|
||||
`INSERT INTO "StepRuns" (
|
||||
@@ -357,6 +377,15 @@ test('serves an authenticated Run through one real SQLite authority and durable
|
||||
fs.chmodSync(root, 0o700);
|
||||
t.after(() => fs.rmSync(root, { recursive: true, force: true }));
|
||||
const databasePath = path.join(root, 'qinglong3.sqlite');
|
||||
const artifactRoot = path.join(root, 'artifacts');
|
||||
const artifactShard = path.join(artifactRoot, 'aa');
|
||||
fs.mkdirSync(artifactShard, { recursive: true, mode: 0o700 });
|
||||
fs.chmodSync(artifactRoot, 0o700);
|
||||
fs.chmodSync(artifactShard, 0o700);
|
||||
const logContent = Buffer.from('local-api-log-line\n', 'utf8');
|
||||
const logPath = path.join(artifactShard, `${LOG_ARTIFACT_ID}.log`);
|
||||
fs.writeFileSync(logPath, logContent, { mode: 0o600 });
|
||||
fs.chmodSync(logPath, 0o600);
|
||||
const keyringDirectory = path.join(root, 'owner-pepper');
|
||||
fs.mkdirSync(keyringDirectory, { mode: 0o700 });
|
||||
const summary = provisionLocalOwnerPepperKey({
|
||||
@@ -398,6 +427,15 @@ test('serves an authenticated Run through one real SQLite authority and durable
|
||||
stepRuns: await runtime.stepRunReader(),
|
||||
runCancellation: await runtime.runCancellationRepository(),
|
||||
taskStart: await runtime.taskStartRepository(),
|
||||
runAttemptLogRead: new RunAttemptLogReadService(
|
||||
runtime.runRepository,
|
||||
new LocalRunAttemptLogRangeReader(artifactRoot),
|
||||
{
|
||||
executorType: 'local_process',
|
||||
artifactIdPattern: /^local-[a-f0-9]{30}$/,
|
||||
maximumReadBytes: 32 * 1024,
|
||||
},
|
||||
),
|
||||
taskDefinitions: runtime.taskDefinitions,
|
||||
apiCredentials: runtime.apiCredentials,
|
||||
ownerPepper: runtime.ownerPepper,
|
||||
@@ -572,12 +610,29 @@ test('serves an authenticated Run through one real SQLite authority and durable
|
||||
});
|
||||
assert.equal(JSON.stringify(steps).includes('private'), false);
|
||||
|
||||
const log = await request(
|
||||
port,
|
||||
`Bearer ${TOKEN}`,
|
||||
`/api/v3/projects/default/runs/${RUN_ID}/attempts/${ATTEMPT_ID}/log?offset=0&length=8`,
|
||||
);
|
||||
assert.equal(log.statusCode, 200);
|
||||
assert.equal(log.body.schema, 'qinglong/run-attempt-log-read-result@v1');
|
||||
assert.equal(log.body.status, 'available');
|
||||
assert.equal(log.body.encoding, 'base64');
|
||||
assert.equal(Buffer.from(log.body.content, 'base64').toString(), 'local-ap');
|
||||
assert.deepEqual(log.body.range, {
|
||||
start: 0,
|
||||
endExclusive: 8,
|
||||
totalBytes: logContent.byteLength,
|
||||
nextOffset: 8,
|
||||
});
|
||||
assert.deepEqual(log.body.truncation, { truncated: 'unknown' });
|
||||
|
||||
const cancellationBody = JSON.stringify({
|
||||
schema: 'qinglong/run-cancellation@v1',
|
||||
mutationId: 'cancel-local-api-1',
|
||||
});
|
||||
const cancellationPath =
|
||||
`/api/v3/projects/default/runs/${RUN_ID}/cancellation`;
|
||||
const cancellationPath = `/api/v3/projects/default/runs/${RUN_ID}/cancellation`;
|
||||
const cancellationOptions = {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
@@ -623,7 +678,7 @@ test('serves an authenticated Run through one real SQLite authority and durable
|
||||
WHERE operation_id IN (
|
||||
'run.get', 'run.list', 'run.events.list', 'run.steps.list',
|
||||
'run.cancel', 'task.get', 'task.list'
|
||||
, 'task.start'
|
||||
, 'task.start', 'run.log.read'
|
||||
)
|
||||
ORDER BY operation_id, outcome`,
|
||||
)
|
||||
@@ -636,6 +691,7 @@ test('serves an authenticated Run through one real SQLite authority and durable
|
||||
'run.get:allowed',
|
||||
'run.get:authentication_rejected',
|
||||
'run.list:allowed',
|
||||
'run.log.read:allowed',
|
||||
'run.steps.list:allowed',
|
||||
'task.get:allowed',
|
||||
'task.get:allowed',
|
||||
|
||||
@@ -455,11 +455,28 @@ export async function bootstrapLocalApplication(
|
||||
});
|
||||
|
||||
if (options.productSurface) {
|
||||
const [stepRuns, runCancellation, taskStart] = await Promise.all([
|
||||
const [
|
||||
stepRuns,
|
||||
runCancellation,
|
||||
taskStart,
|
||||
{ LocalRunAttemptLogRangeReader },
|
||||
{ RunAttemptLogReadService },
|
||||
] = await Promise.all([
|
||||
storage.stepRunReader(),
|
||||
storage.runCancellationRepository(),
|
||||
storage.taskStartRepository(),
|
||||
import('@qinglong/local-execution/artifact-read'),
|
||||
import('@qinglong/runtime-core/run-attempt-log-read'),
|
||||
]);
|
||||
const runAttemptLogRead = new RunAttemptLogReadService(
|
||||
storage.runs,
|
||||
new LocalRunAttemptLogRangeReader(options.artifactRoot),
|
||||
{
|
||||
executorType: 'local_process',
|
||||
artifactIdPattern: /^local-[a-f0-9]{30}$/,
|
||||
maximumReadBytes: 32 * 1024,
|
||||
},
|
||||
);
|
||||
productSurfaceLifecycle = await options.productSurface.start(
|
||||
Object.freeze({
|
||||
profile: options.profile,
|
||||
@@ -467,6 +484,7 @@ export async function bootstrapLocalApplication(
|
||||
stepRuns,
|
||||
runCancellation,
|
||||
taskStart,
|
||||
runAttemptLogRead,
|
||||
taskDefinitions: storage.taskDefinitions,
|
||||
apiCredentials: storage.apiCredentials,
|
||||
ownerPepper: storage.ownerPepper,
|
||||
|
||||
@@ -18,6 +18,10 @@ import type { PluginPackageRecoveryCycleResult } from '@qinglong/runtime-core/pl
|
||||
import type { PluginPackageAutomationPublicationRecoveryCycleResult } from '@qinglong/runtime-core/plugin-package-automation-publication';
|
||||
import type { PluginPackageTaskPublicationRecoveryCycleResult } from '@qinglong/runtime-core/plugin-package-task-publication';
|
||||
import type { ProjectToolDefinitionSnapshotRecoveryCycleResult } from '@qinglong/runtime-core/project-tool-definition-snapshot';
|
||||
import type {
|
||||
RunAttemptLogReadRequest,
|
||||
RunAttemptLogReadResult,
|
||||
} from '@qinglong/runtime-core/run-attempt-log-read';
|
||||
|
||||
export type LocalApplicationProfile = 'edge' | 'standalone';
|
||||
|
||||
@@ -59,6 +63,11 @@ export interface LocalApplicationProductSurfaceAuthority {
|
||||
ReadyFreshStorage['taskDefinitions'],
|
||||
'findCurrentTaskDefinition' | 'listTaskDefinitions'
|
||||
>;
|
||||
readonly runAttemptLogRead: Readonly<{
|
||||
read(
|
||||
request: Readonly<RunAttemptLogReadRequest>,
|
||||
): Promise<RunAttemptLogReadResult>;
|
||||
}>;
|
||||
readonly apiCredentials: ReadyFreshStorage['apiCredentials'];
|
||||
readonly ownerPepper: ReadyFreshStorage['ownerPepper'];
|
||||
readonly projectPolicy: ReadyFreshStorage['projectPolicy'];
|
||||
|
||||
@@ -88,8 +88,7 @@ const RECEIPT_TOKEN = 'A'.repeat(32);
|
||||
const CLEANUP_RUN_ID = '019f70c0-0000-7000-8000-000000000011';
|
||||
const CLEANUP_ATTEMPT_ID = '019f70c0-0000-7000-8000-000000000012';
|
||||
const WORKFLOW_CANCELLATION_CREDENTIAL_ID = 'application-workflow-owner';
|
||||
const WORKFLOW_CANCELLATION_PEPPER_KEY_ID =
|
||||
'application-workflow-owner-v1';
|
||||
const WORKFLOW_CANCELLATION_PEPPER_KEY_ID = 'application-workflow-owner-v1';
|
||||
const WORKFLOW_CANCELLATION_PEPPER_BYTES = Buffer.alloc(32, 141);
|
||||
const WORKFLOW_CANCELLATION_SECRET = Buffer.alloc(32, 142).toString(
|
||||
'base64url',
|
||||
@@ -322,8 +321,7 @@ function promptResourceConfiguration() {
|
||||
const prefix = 'private router model output:';
|
||||
const minimumOutputBytes = Buffer.byteLength(prefix, 'utf8');
|
||||
const rawOutputBytes =
|
||||
process.env.QL3_PROMPT_RESOURCE_OUTPUT_BYTES ??
|
||||
String(minimumOutputBytes);
|
||||
process.env.QL3_PROMPT_RESOURCE_OUTPUT_BYTES ?? String(minimumOutputBytes);
|
||||
const outputBytes = Number(rawOutputBytes);
|
||||
if (
|
||||
!Number.isSafeInteger(outputBytes) ||
|
||||
@@ -338,8 +336,7 @@ function promptResourceConfiguration() {
|
||||
profile,
|
||||
output: prefix + 'x'.repeat(outputBytes - Buffer.byteLength(prefix)),
|
||||
outputBytes,
|
||||
resourceProbe:
|
||||
process.env.QL3_PROMPT_RESOURCE_OUTPUT_BYTES !== undefined,
|
||||
resourceProbe: process.env.QL3_PROMPT_RESOURCE_OUTPUT_BYTES !== undefined,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1263,7 +1260,10 @@ test('executes one active Package Prompt through local AI composition with conte
|
||||
promptOutputKeys: {
|
||||
async active() {
|
||||
metrics.keyLoads += 1;
|
||||
return { keyId: 'edge-prompt-output-key-1', key: Buffer.alloc(32, 7) };
|
||||
return {
|
||||
keyId: 'edge-prompt-output-key-1',
|
||||
key: Buffer.alloc(32, 7),
|
||||
};
|
||||
},
|
||||
async resolve(keyId) {
|
||||
metrics.keyResolutions += 1;
|
||||
@@ -1274,7 +1274,10 @@ test('executes one active Package Prompt through local AI composition with conte
|
||||
promptOutputRead: {
|
||||
authorizer: {
|
||||
async authorize(request) {
|
||||
assert.equal(request.projectId, 'project-application-prompt-resource');
|
||||
assert.equal(
|
||||
request.projectId,
|
||||
'project-application-prompt-resource',
|
||||
);
|
||||
assert.equal(request.principal.subject.id, 'edge-resource-owner');
|
||||
return { effect: 'allow' };
|
||||
},
|
||||
@@ -1376,7 +1379,10 @@ test('executes one active Package Prompt through local AI composition with conte
|
||||
assert.equal(recoveredByRequest.result.text, metrics.privateOutput);
|
||||
assert.deepEqual(recoveredByRequest.reference, durableFirst.outputArtifact);
|
||||
assert.equal(metrics.keyResolutions, 2);
|
||||
assert.equal(metrics.lastResolvedKey.every((byte) => byte === 0), true);
|
||||
assert.equal(
|
||||
metrics.lastResolvedKey.every((byte) => byte === 0),
|
||||
true,
|
||||
);
|
||||
const durableStorageAfter = sqliteStorageSnapshot(value.targetPath);
|
||||
const durableStorageGrowth = sqliteStorageGrowth(
|
||||
durableStorageBefore,
|
||||
@@ -1450,9 +1456,8 @@ test('executes one active Package Prompt through local AI composition with conte
|
||||
)
|
||||
.get(durableFirst.admission.invocationId),
|
||||
},
|
||||
integrityCheck: reader
|
||||
.prepare('PRAGMA integrity_check')
|
||||
.get().integrity_check,
|
||||
integrityCheck: reader.prepare('PRAGMA integrity_check').get()
|
||||
.integrity_check,
|
||||
};
|
||||
} finally {
|
||||
reader.close();
|
||||
@@ -1532,8 +1537,7 @@ test('executes one active Package Prompt through local AI composition with conte
|
||||
databaseLogicalWriteAmplificationPermille,
|
||||
databaseAllocatedWriteAmplificationPermille,
|
||||
walWriteAmplificationPermille,
|
||||
journalMode:
|
||||
resource.profile === 'edge' ? 'delete' : 'wal',
|
||||
journalMode: resource.profile === 'edge' ? 'delete' : 'wal',
|
||||
providerCalls: metrics.providerCalls,
|
||||
keyLoads: metrics.keyLoads,
|
||||
keyResolutions: metrics.keyResolutions,
|
||||
@@ -1700,6 +1704,7 @@ test('starts an optional product surface after recovery and drains it before own
|
||||
typeof authority.taskDefinitions.listTaskDefinitions,
|
||||
'function',
|
||||
);
|
||||
assert.equal(typeof authority.runAttemptLogRead.read, 'function');
|
||||
assert.equal(typeof authority.apiCredentials.resolve, 'function');
|
||||
assert.equal(typeof authority.ownerPepper.resolveKey, 'function');
|
||||
assert.equal(typeof authority.projectPolicy.resolve, 'function');
|
||||
|
||||
@@ -28,6 +28,11 @@
|
||||
"require": "./dist/dispatch/index.js",
|
||||
"default": "./dist/dispatch/index.js"
|
||||
},
|
||||
"./artifact-read": {
|
||||
"types": "./dist/artifact-read/index.d.ts",
|
||||
"require": "./dist/artifact-read/index.js",
|
||||
"default": "./dist/artifact-read/index.js"
|
||||
},
|
||||
"./scheduler": {
|
||||
"types": "./dist/scheduler/index.d.ts",
|
||||
"require": "./dist/scheduler/index.js",
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
export * from './localRunAttemptLogRangeReader';
|
||||
@@ -0,0 +1,285 @@
|
||||
import { constants, type Stats } from 'node:fs';
|
||||
import fs, { type FileHandle } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
import {
|
||||
normalizeRunAttemptLogReadRange,
|
||||
type RunAttemptLogRangeReader,
|
||||
type RunAttemptLogRangeReadResult,
|
||||
type RunAttemptLogReadIdentity,
|
||||
type RunAttemptLogReadRange,
|
||||
type RunAttemptLogTruncationView,
|
||||
} from '@qinglong/runtime-core/run-attempt-log-read';
|
||||
|
||||
const LOCAL_ARTIFACT_ID = /^local-[a-f0-9]{30}$/;
|
||||
const MAXIMUM_ARTIFACT_BYTES = 1024 * 1024 * 1024;
|
||||
const MAXIMUM_FACT_BYTES = 1024;
|
||||
|
||||
export class LocalRunAttemptLogRangeReadError extends Error {
|
||||
constructor(
|
||||
readonly reason:
|
||||
| 'invalid_configuration'
|
||||
| 'unsafe_path'
|
||||
| 'integrity_mismatch',
|
||||
options?: ErrorOptions,
|
||||
) {
|
||||
super(`Local Run Attempt log range read failed: ${reason}`, options);
|
||||
this.name = 'LocalRunAttemptLogRangeReadError';
|
||||
}
|
||||
}
|
||||
|
||||
function isCode(error: unknown, code: string): boolean {
|
||||
return (
|
||||
!!error &&
|
||||
typeof error === 'object' &&
|
||||
'code' in error &&
|
||||
(error as { code?: unknown }).code === code
|
||||
);
|
||||
}
|
||||
|
||||
function currentUid(): number | undefined {
|
||||
return typeof process.getuid === 'function' ? process.getuid() : undefined;
|
||||
}
|
||||
|
||||
function root(value: string): string {
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
!path.isAbsolute(value) ||
|
||||
path.parse(value).root === value ||
|
||||
value.includes('\0') ||
|
||||
Buffer.byteLength(value, 'utf8') > 4096
|
||||
) {
|
||||
throw new LocalRunAttemptLogRangeReadError('invalid_configuration');
|
||||
}
|
||||
return path.resolve(value);
|
||||
}
|
||||
|
||||
function identity(
|
||||
value: Readonly<RunAttemptLogReadIdentity>,
|
||||
): Readonly<RunAttemptLogReadIdentity> {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== 'object' ||
|
||||
Array.isArray(value) ||
|
||||
!LOCAL_ARTIFACT_ID.test(value.logArtifactId)
|
||||
) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function assertOwnedDirectory(stat: Stats): void {
|
||||
const uid = currentUid();
|
||||
if (
|
||||
!stat.isDirectory() ||
|
||||
stat.isSymbolicLink() ||
|
||||
(stat.mode & 0o777) !== 0o700 ||
|
||||
(uid !== undefined && stat.uid !== uid)
|
||||
) {
|
||||
throw new LocalRunAttemptLogRangeReadError('unsafe_path');
|
||||
}
|
||||
}
|
||||
|
||||
function assertOwnedFile(stat: Stats): void {
|
||||
const uid = currentUid();
|
||||
if (
|
||||
!stat.isFile() ||
|
||||
stat.nlink !== 1 ||
|
||||
(stat.mode & 0o777) !== 0o600 ||
|
||||
(uid !== undefined && stat.uid !== uid) ||
|
||||
!Number.isSafeInteger(stat.size) ||
|
||||
stat.size < 0 ||
|
||||
stat.size > MAXIMUM_ARTIFACT_BYTES
|
||||
) {
|
||||
throw new LocalRunAttemptLogRangeReadError('unsafe_path');
|
||||
}
|
||||
}
|
||||
|
||||
async function optionalPrivateDirectory(directory: string): Promise<boolean> {
|
||||
try {
|
||||
assertOwnedDirectory(await fs.lstat(directory));
|
||||
return true;
|
||||
} catch (error) {
|
||||
if (isCode(error, 'ENOENT')) return false;
|
||||
if (error instanceof LocalRunAttemptLogRangeReadError) throw error;
|
||||
throw new LocalRunAttemptLogRangeReadError('unsafe_path', { cause: error });
|
||||
}
|
||||
}
|
||||
|
||||
async function openPrivateFile(
|
||||
filePath: string,
|
||||
): Promise<FileHandle | undefined> {
|
||||
try {
|
||||
return await fs.open(
|
||||
filePath,
|
||||
constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0),
|
||||
);
|
||||
} catch (error) {
|
||||
if (isCode(error, 'ENOENT')) return undefined;
|
||||
throw new LocalRunAttemptLogRangeReadError('unsafe_path', { cause: error });
|
||||
}
|
||||
}
|
||||
|
||||
function exactFact(
|
||||
value: unknown,
|
||||
expected: Readonly<RunAttemptLogReadIdentity>,
|
||||
): Readonly<RunAttemptLogTruncationView> {
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
const fact = value as Record<string, unknown>;
|
||||
const keys = Object.keys(fact).sort();
|
||||
if (
|
||||
keys.join(',') !==
|
||||
'attemptId,logArtifactId,maximumBytes,observedAtMs,quotaReached,runId,schemaVersion' ||
|
||||
fact.schemaVersion !== 1 ||
|
||||
fact.runId !== expected.runId ||
|
||||
fact.attemptId !== expected.attemptId ||
|
||||
fact.logArtifactId !== expected.logArtifactId ||
|
||||
!Number.isSafeInteger(fact.maximumBytes) ||
|
||||
Number(fact.maximumBytes) < 64 * 1024 ||
|
||||
Number(fact.maximumBytes) > MAXIMUM_ARTIFACT_BYTES ||
|
||||
typeof fact.quotaReached !== 'boolean' ||
|
||||
!Number.isSafeInteger(fact.observedAtMs) ||
|
||||
Number(fact.observedAtMs) < 0
|
||||
) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
return Object.freeze({
|
||||
truncated: fact.quotaReached,
|
||||
maximumBytes: fact.maximumBytes as number,
|
||||
observedAtMs: fact.observedAtMs as number,
|
||||
});
|
||||
}
|
||||
|
||||
async function readTruncationFact(
|
||||
directory: string,
|
||||
expected: Readonly<RunAttemptLogReadIdentity>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Readonly<RunAttemptLogTruncationView>> {
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
const factPath = path.join(
|
||||
directory,
|
||||
`.${expected.logArtifactId}.log.truncated.json`,
|
||||
);
|
||||
const handle = await openPrivateFile(factPath);
|
||||
if (!handle) return Object.freeze({ truncated: 'unknown' as const });
|
||||
try {
|
||||
const before = await handle.stat();
|
||||
assertOwnedFile(before);
|
||||
if (before.size < 2 || before.size > MAXIMUM_FACT_BYTES) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
const content = Buffer.allocUnsafe(before.size);
|
||||
let read = 0;
|
||||
while (read < content.byteLength) {
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
const result = await handle.read(
|
||||
content,
|
||||
read,
|
||||
content.byteLength - read,
|
||||
read,
|
||||
);
|
||||
if (result.bytesRead < 1) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
read += result.bytesRead;
|
||||
}
|
||||
const after = await handle.stat();
|
||||
assertOwnedFile(after);
|
||||
if (
|
||||
after.dev !== before.dev ||
|
||||
after.ino !== before.ino ||
|
||||
after.size !== before.size
|
||||
) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
try {
|
||||
const text = new TextDecoder('utf-8', { fatal: true }).decode(content);
|
||||
return exactFact(JSON.parse(text), expected);
|
||||
} catch (error) {
|
||||
if (error instanceof LocalRunAttemptLogRangeReadError) throw error;
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch', {
|
||||
cause: error,
|
||||
});
|
||||
} finally {
|
||||
content.fill(0);
|
||||
}
|
||||
} finally {
|
||||
await handle.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
export class LocalRunAttemptLogRangeReader implements RunAttemptLogRangeReader {
|
||||
private readonly root: string;
|
||||
|
||||
constructor(artifactRoot: string) {
|
||||
this.root = root(artifactRoot);
|
||||
}
|
||||
|
||||
async read(
|
||||
rawIdentity: Readonly<RunAttemptLogReadIdentity>,
|
||||
rawRange: Readonly<RunAttemptLogReadRange>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<RunAttemptLogRangeReadResult> {
|
||||
const expected = identity(rawIdentity);
|
||||
const range = normalizeRunAttemptLogReadRange(rawRange);
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
if (!(await optionalPrivateDirectory(this.root))) {
|
||||
return Object.freeze({ status: 'missing' as const });
|
||||
}
|
||||
const directory = path.join(
|
||||
this.root,
|
||||
expected.logArtifactId.slice('local-'.length, 'local-'.length + 2),
|
||||
);
|
||||
if (!(await optionalPrivateDirectory(directory))) {
|
||||
return Object.freeze({ status: 'missing' as const });
|
||||
}
|
||||
const target = path.join(directory, `${expected.logArtifactId}.log`);
|
||||
const handle = await openPrivateFile(target);
|
||||
if (!handle) return Object.freeze({ status: 'missing' as const });
|
||||
try {
|
||||
const before = await handle.stat();
|
||||
assertOwnedFile(before);
|
||||
const start = Math.min(range.offset, before.size);
|
||||
const expectedBytes = Math.min(range.length, before.size - start);
|
||||
const content = Buffer.allocUnsafe(expectedBytes);
|
||||
let read = 0;
|
||||
while (read < expectedBytes) {
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
const result = await handle.read(
|
||||
content,
|
||||
read,
|
||||
expectedBytes - read,
|
||||
start + read,
|
||||
);
|
||||
if (result.bytesRead < 1) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
read += result.bytesRead;
|
||||
}
|
||||
const after = await handle.stat();
|
||||
assertOwnedFile(after);
|
||||
if (
|
||||
after.dev !== before.dev ||
|
||||
after.ino !== before.ino ||
|
||||
after.size < before.size
|
||||
) {
|
||||
throw new LocalRunAttemptLogRangeReadError('integrity_mismatch');
|
||||
}
|
||||
const endExclusive = start + content.byteLength;
|
||||
const truncation = await readTruncationFact(directory, expected, signal);
|
||||
return Object.freeze({
|
||||
status: 'available' as const,
|
||||
content,
|
||||
start,
|
||||
endExclusive,
|
||||
totalBytes: before.size,
|
||||
...(endExclusive < before.size ? { nextOffset: endExclusive } : {}),
|
||||
truncation,
|
||||
});
|
||||
} finally {
|
||||
await handle.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,158 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs/promises');
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
const { test } = require('node:test');
|
||||
|
||||
const {
|
||||
LocalRunAttemptLogRangeReadError,
|
||||
LocalRunAttemptLogRangeReader,
|
||||
} = require('../dist/artifact-read/localRunAttemptLogRangeReader.js');
|
||||
|
||||
const artifactId = `local-${'a'.repeat(30)}`;
|
||||
const identity = Object.freeze({
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
logArtifactId: artifactId,
|
||||
});
|
||||
|
||||
async function fixture(t, content = Buffer.from('0123456789')) {
|
||||
const parent = await fs.mkdtemp(path.join(os.tmpdir(), 'ql3-log-read-'));
|
||||
t.after(() => fs.rm(parent, { recursive: true, force: true }));
|
||||
const root = path.join(parent, 'artifacts');
|
||||
const shard = path.join(root, 'aa');
|
||||
await fs.mkdir(shard, { recursive: true, mode: 0o700 });
|
||||
await fs.chmod(root, 0o700);
|
||||
await fs.chmod(shard, 0o700);
|
||||
const log = path.join(shard, `${artifactId}.log`);
|
||||
await fs.writeFile(log, content, { mode: 0o600 });
|
||||
await fs.chmod(log, 0o600);
|
||||
return { parent, root, shard, log };
|
||||
}
|
||||
|
||||
async function fact(shard, overrides = {}) {
|
||||
const file = path.join(shard, `.${artifactId}.log.truncated.json`);
|
||||
await fs.writeFile(
|
||||
file,
|
||||
JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
runId: identity.runId,
|
||||
attemptId: identity.attemptId,
|
||||
logArtifactId: identity.logArtifactId,
|
||||
maximumBytes: 64 * 1024,
|
||||
quotaReached: false,
|
||||
observedAtMs: 9,
|
||||
...overrides,
|
||||
}),
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
await fs.chmod(file, 0o600);
|
||||
return file;
|
||||
}
|
||||
|
||||
test('reads one bounded private-file snapshot and canonical truncation fact', async (t) => {
|
||||
const value = await fixture(t);
|
||||
await fact(value.shard, { quotaReached: true });
|
||||
const result = await new LocalRunAttemptLogRangeReader(value.root).read(
|
||||
identity,
|
||||
{ offset: 2, length: 4 },
|
||||
);
|
||||
assert.equal(result.status, 'available');
|
||||
assert.equal(Buffer.from(result.content).toString(), '2345');
|
||||
assert.deepEqual(
|
||||
{
|
||||
start: result.start,
|
||||
endExclusive: result.endExclusive,
|
||||
totalBytes: result.totalBytes,
|
||||
nextOffset: result.nextOffset,
|
||||
truncation: result.truncation,
|
||||
},
|
||||
{
|
||||
start: 2,
|
||||
endExclusive: 6,
|
||||
totalBytes: 10,
|
||||
nextOffset: 6,
|
||||
truncation: {
|
||||
truncated: true,
|
||||
maximumBytes: 64 * 1024,
|
||||
observedAtMs: 9,
|
||||
},
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('returns unknown truncation and a stable empty range beyond the snapshot', async (t) => {
|
||||
const value = await fixture(t);
|
||||
const result = await new LocalRunAttemptLogRangeReader(value.root).read(
|
||||
identity,
|
||||
{ offset: 99, length: 4 },
|
||||
);
|
||||
assert.equal(result.status, 'available');
|
||||
assert.equal(result.content.byteLength, 0);
|
||||
assert.equal(result.start, 10);
|
||||
assert.equal(result.endExclusive, 10);
|
||||
assert.equal(result.totalBytes, 10);
|
||||
assert.equal(result.nextOffset, undefined);
|
||||
assert.deepEqual(result.truncation, { truncated: 'unknown' });
|
||||
});
|
||||
|
||||
test('treats absent root, shard and log as missing', async (t) => {
|
||||
const parent = await fs.mkdtemp(path.join(os.tmpdir(), 'ql3-log-missing-'));
|
||||
t.after(() => fs.rm(parent, { recursive: true, force: true }));
|
||||
const root = path.join(parent, 'artifacts');
|
||||
const reader = new LocalRunAttemptLogRangeReader(root);
|
||||
assert.deepEqual(await reader.read(identity, { offset: 0, length: 1 }), {
|
||||
status: 'missing',
|
||||
});
|
||||
await fs.mkdir(root, { mode: 0o700 });
|
||||
assert.deepEqual(await reader.read(identity, { offset: 0, length: 1 }), {
|
||||
status: 'missing',
|
||||
});
|
||||
const shard = path.join(root, 'aa');
|
||||
await fs.mkdir(shard, { mode: 0o700 });
|
||||
assert.deepEqual(await reader.read(identity, { offset: 0, length: 1 }), {
|
||||
status: 'missing',
|
||||
});
|
||||
});
|
||||
|
||||
test('fails closed for symlink targets and widened file permissions', async (t) => {
|
||||
const symlink = await fixture(t);
|
||||
await fs.rm(symlink.log);
|
||||
await fs.symlink(path.join(symlink.parent, 'outside'), symlink.log);
|
||||
await assert.rejects(
|
||||
new LocalRunAttemptLogRangeReader(symlink.root).read(identity, {
|
||||
offset: 0,
|
||||
length: 1,
|
||||
}),
|
||||
LocalRunAttemptLogRangeReadError,
|
||||
);
|
||||
|
||||
const widened = await fixture(t);
|
||||
await fs.chmod(widened.log, 0o644);
|
||||
await assert.rejects(
|
||||
new LocalRunAttemptLogRangeReader(widened.root).read(identity, {
|
||||
offset: 0,
|
||||
length: 1,
|
||||
}),
|
||||
LocalRunAttemptLogRangeReadError,
|
||||
);
|
||||
});
|
||||
|
||||
test('fails closed for truncation identity drift and an aborted request', async (t) => {
|
||||
const value = await fixture(t);
|
||||
await fact(value.shard, { attemptId: 'attempt_other' });
|
||||
const reader = new LocalRunAttemptLogRangeReader(value.root);
|
||||
await assert.rejects(
|
||||
reader.read(identity, { offset: 0, length: 1 }),
|
||||
(error) =>
|
||||
error instanceof LocalRunAttemptLogRangeReadError &&
|
||||
error.reason === 'integrity_mismatch',
|
||||
);
|
||||
const abort = new AbortController();
|
||||
abort.abort(new Error('cancelled'));
|
||||
await assert.rejects(
|
||||
reader.read(identity, { offset: 0, length: 1 }, abort.signal),
|
||||
/cancelled/,
|
||||
);
|
||||
});
|
||||
@@ -232,6 +232,9 @@
|
||||
],
|
||||
"bounded-run-step-list-projection": [
|
||||
"dist/run/projection/boundedRunStepListProjection.d.ts"
|
||||
],
|
||||
"run-attempt-log-read": [
|
||||
"dist/run/log-read/runAttemptLogRead.d.ts"
|
||||
]
|
||||
}
|
||||
},
|
||||
@@ -281,6 +284,11 @@
|
||||
"require": "./dist/run/projection/boundedRunStepListProjection.js",
|
||||
"default": "./dist/run/projection/boundedRunStepListProjection.js"
|
||||
},
|
||||
"./run-attempt-log-read": {
|
||||
"types": "./dist/run/log-read/runAttemptLogRead.d.ts",
|
||||
"require": "./dist/run/log-read/runAttemptLogRead.js",
|
||||
"default": "./dist/run/log-read/runAttemptLogRead.js"
|
||||
},
|
||||
"./task-definition": {
|
||||
"types": "./dist/task-definition/taskDefinition.d.ts",
|
||||
"require": "./dist/task-definition/taskDefinition.js",
|
||||
|
||||
@@ -0,0 +1,343 @@
|
||||
import { RUN_ATTEMPT_STATUSES, type RunAttemptStatus } from '../run';
|
||||
import type { RunRepositoryReader } from '../runRepository';
|
||||
|
||||
export const MAX_RUN_ATTEMPT_LOG_READ_BYTES = 256 * 1024;
|
||||
|
||||
const ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/;
|
||||
const TERMINAL_ATTEMPT_STATUSES = new Set<RunAttemptStatus>([
|
||||
'succeeded',
|
||||
'failed',
|
||||
'cancelled',
|
||||
'timed_out',
|
||||
'lost',
|
||||
]);
|
||||
|
||||
export interface RunAttemptLogReadRange {
|
||||
readonly offset: number;
|
||||
readonly length: number;
|
||||
}
|
||||
|
||||
export interface RunAttemptLogReadIdentity {
|
||||
readonly projectId: string;
|
||||
readonly runId: string;
|
||||
readonly attemptId: string;
|
||||
readonly logArtifactId: string;
|
||||
}
|
||||
|
||||
export interface RunAttemptLogTruncationView {
|
||||
readonly truncated: boolean | 'unknown';
|
||||
readonly maximumBytes?: number;
|
||||
readonly observedAtMs?: number;
|
||||
}
|
||||
|
||||
export type RunAttemptLogRangeReadResult =
|
||||
| Readonly<{ readonly status: 'missing' }>
|
||||
| Readonly<{
|
||||
readonly status: 'available';
|
||||
readonly content: Uint8Array;
|
||||
readonly start: number;
|
||||
readonly endExclusive: number;
|
||||
readonly totalBytes: number;
|
||||
readonly nextOffset?: number;
|
||||
readonly truncation: Readonly<RunAttemptLogTruncationView>;
|
||||
}>;
|
||||
|
||||
export interface RunAttemptLogRangeReader {
|
||||
read(
|
||||
identity: Readonly<RunAttemptLogReadIdentity>,
|
||||
range: Readonly<RunAttemptLogReadRange>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<RunAttemptLogRangeReadResult>;
|
||||
}
|
||||
|
||||
export interface RunAttemptLogReadRequest {
|
||||
readonly projectId: string;
|
||||
readonly runId: string;
|
||||
readonly attemptId: string;
|
||||
readonly range: Readonly<RunAttemptLogReadRange>;
|
||||
readonly signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export type RunAttemptLogReadResult =
|
||||
| Readonly<{ readonly status: 'not_found' }>
|
||||
| Readonly<{
|
||||
readonly status: 'pending';
|
||||
readonly projectId: string;
|
||||
readonly runId: string;
|
||||
readonly attemptId: string;
|
||||
readonly logArtifactId?: string;
|
||||
}>
|
||||
| (Readonly<RunAttemptLogReadIdentity> &
|
||||
Readonly<{ readonly status: 'missing' }>)
|
||||
| (Readonly<RunAttemptLogReadIdentity> &
|
||||
Extract<RunAttemptLogRangeReadResult, { readonly status: 'available' }>);
|
||||
|
||||
export interface RunAttemptLogReadServiceOptions {
|
||||
readonly executorType: 'local_process' | 'remote_worker';
|
||||
readonly artifactIdPattern: RegExp;
|
||||
readonly maximumReadBytes: number;
|
||||
readonly activeMissingIsPending?: boolean;
|
||||
}
|
||||
|
||||
export class InvalidRunAttemptLogReadError extends TypeError {
|
||||
constructor(message: string) {
|
||||
super(`Run Attempt log read is invalid: ${message}`);
|
||||
this.name = 'InvalidRunAttemptLogReadError';
|
||||
}
|
||||
}
|
||||
|
||||
export class RunAttemptLogReadUnavailableError extends Error {
|
||||
constructor(options?: ErrorOptions) {
|
||||
super('Run Attempt log read is unavailable', options);
|
||||
this.name = 'RunAttemptLogReadUnavailableError';
|
||||
}
|
||||
}
|
||||
|
||||
function canonicalId(name: string, value: unknown): string {
|
||||
if (typeof value !== 'string' || !ID_PATTERN.test(value)) {
|
||||
throw new InvalidRunAttemptLogReadError(`${name} is invalid`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function exactKeys(
|
||||
value: object,
|
||||
required: readonly string[],
|
||||
optional: readonly string[],
|
||||
name: string,
|
||||
): void {
|
||||
const keys = Object.keys(value);
|
||||
const allowed = new Set([...required, ...optional]);
|
||||
if (
|
||||
required.some((key) => !Object.hasOwn(value, key)) ||
|
||||
keys.some((key) => !allowed.has(key))
|
||||
) {
|
||||
throw new InvalidRunAttemptLogReadError(`${name} shape is invalid`);
|
||||
}
|
||||
}
|
||||
|
||||
export function normalizeRunAttemptLogReadRange(
|
||||
value: Readonly<RunAttemptLogReadRange>,
|
||||
maximumReadBytes = MAX_RUN_ATTEMPT_LOG_READ_BYTES,
|
||||
): Readonly<RunAttemptLogReadRange> {
|
||||
if (
|
||||
!Number.isSafeInteger(maximumReadBytes) ||
|
||||
maximumReadBytes < 1 ||
|
||||
maximumReadBytes > MAX_RUN_ATTEMPT_LOG_READ_BYTES
|
||||
) {
|
||||
throw new InvalidRunAttemptLogReadError('maximum read bytes is invalid');
|
||||
}
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
||||
throw new InvalidRunAttemptLogReadError('range is invalid');
|
||||
}
|
||||
exactKeys(value, ['length', 'offset'], [], 'range');
|
||||
if (!Number.isSafeInteger(value.offset) || value.offset < 0) {
|
||||
throw new InvalidRunAttemptLogReadError('offset is invalid');
|
||||
}
|
||||
if (
|
||||
!Number.isSafeInteger(value.length) ||
|
||||
value.length < 1 ||
|
||||
value.length > maximumReadBytes
|
||||
) {
|
||||
throw new InvalidRunAttemptLogReadError('length is invalid');
|
||||
}
|
||||
return Object.freeze({ offset: value.offset, length: value.length });
|
||||
}
|
||||
|
||||
function prepareOptions(
|
||||
options: RunAttemptLogReadServiceOptions,
|
||||
): Readonly<RunAttemptLogReadServiceOptions> {
|
||||
if (
|
||||
!options ||
|
||||
typeof options !== 'object' ||
|
||||
Array.isArray(options) ||
|
||||
(options.executorType !== 'local_process' &&
|
||||
options.executorType !== 'remote_worker') ||
|
||||
!(options.artifactIdPattern instanceof RegExp) ||
|
||||
options.artifactIdPattern.global ||
|
||||
options.artifactIdPattern.sticky ||
|
||||
(options.activeMissingIsPending !== undefined &&
|
||||
typeof options.activeMissingIsPending !== 'boolean')
|
||||
) {
|
||||
throw new InvalidRunAttemptLogReadError('service options are invalid');
|
||||
}
|
||||
exactKeys(
|
||||
options,
|
||||
['artifactIdPattern', 'executorType', 'maximumReadBytes'],
|
||||
['activeMissingIsPending'],
|
||||
'service options',
|
||||
);
|
||||
normalizeRunAttemptLogReadRange(
|
||||
{ offset: 0, length: options.maximumReadBytes },
|
||||
options.maximumReadBytes,
|
||||
);
|
||||
return Object.freeze({ ...options });
|
||||
}
|
||||
|
||||
function truncation(
|
||||
value: Readonly<RunAttemptLogTruncationView>,
|
||||
): Readonly<RunAttemptLogTruncationView> {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== 'object' ||
|
||||
Array.isArray(value) ||
|
||||
(value.truncated !== true &&
|
||||
value.truncated !== false &&
|
||||
value.truncated !== 'unknown') ||
|
||||
(value.maximumBytes !== undefined &&
|
||||
(!Number.isSafeInteger(value.maximumBytes) || value.maximumBytes < 1)) ||
|
||||
(value.observedAtMs !== undefined &&
|
||||
(!Number.isSafeInteger(value.observedAtMs) || value.observedAtMs < 0)) ||
|
||||
(value.truncated === 'unknown' &&
|
||||
(value.maximumBytes !== undefined || value.observedAtMs !== undefined))
|
||||
) {
|
||||
throw new RunAttemptLogReadUnavailableError();
|
||||
}
|
||||
return Object.freeze({ ...value });
|
||||
}
|
||||
|
||||
function available(
|
||||
identity: Readonly<RunAttemptLogReadIdentity>,
|
||||
range: Readonly<RunAttemptLogReadRange>,
|
||||
result: Extract<
|
||||
RunAttemptLogRangeReadResult,
|
||||
{ readonly status: 'available' }
|
||||
>,
|
||||
): RunAttemptLogReadResult {
|
||||
if (
|
||||
!(result.content instanceof Uint8Array) ||
|
||||
!Number.isSafeInteger(result.start) ||
|
||||
!Number.isSafeInteger(result.endExclusive) ||
|
||||
!Number.isSafeInteger(result.totalBytes) ||
|
||||
result.start !== Math.min(range.offset, result.totalBytes) ||
|
||||
result.endExclusive !== result.start + result.content.byteLength ||
|
||||
result.endExclusive > result.totalBytes ||
|
||||
result.content.byteLength > range.length ||
|
||||
(result.nextOffset === undefined) !==
|
||||
(result.endExclusive === result.totalBytes) ||
|
||||
(result.nextOffset !== undefined &&
|
||||
result.nextOffset !== result.endExclusive)
|
||||
) {
|
||||
throw new RunAttemptLogReadUnavailableError();
|
||||
}
|
||||
return Object.freeze({
|
||||
status: 'available' as const,
|
||||
...identity,
|
||||
content: result.content,
|
||||
start: result.start,
|
||||
endExclusive: result.endExclusive,
|
||||
totalBytes: result.totalBytes,
|
||||
...(result.nextOffset === undefined
|
||||
? {}
|
||||
: { nextOffset: result.nextOffset }),
|
||||
truncation: truncation(result.truncation),
|
||||
});
|
||||
}
|
||||
|
||||
export class RunAttemptLogReadService {
|
||||
private readonly options: Readonly<RunAttemptLogReadServiceOptions>;
|
||||
|
||||
constructor(
|
||||
private readonly runs: Pick<
|
||||
RunRepositoryReader,
|
||||
'findRunById' | 'findAttemptById'
|
||||
>,
|
||||
private readonly reader: RunAttemptLogRangeReader,
|
||||
options: RunAttemptLogReadServiceOptions,
|
||||
) {
|
||||
if (
|
||||
!runs ||
|
||||
typeof runs.findRunById !== 'function' ||
|
||||
typeof runs.findAttemptById !== 'function' ||
|
||||
!reader ||
|
||||
typeof reader.read !== 'function'
|
||||
) {
|
||||
throw new InvalidRunAttemptLogReadError('dependencies are invalid');
|
||||
}
|
||||
this.options = prepareOptions(options);
|
||||
}
|
||||
|
||||
async read(
|
||||
request: Readonly<RunAttemptLogReadRequest>,
|
||||
): Promise<RunAttemptLogReadResult> {
|
||||
if (!request || typeof request !== 'object' || Array.isArray(request)) {
|
||||
throw new InvalidRunAttemptLogReadError('request is invalid');
|
||||
}
|
||||
exactKeys(
|
||||
request,
|
||||
['attemptId', 'projectId', 'range', 'runId'],
|
||||
['signal'],
|
||||
'request',
|
||||
);
|
||||
const projectId = canonicalId('projectId', request.projectId);
|
||||
const runId = canonicalId('runId', request.runId);
|
||||
const attemptId = canonicalId('attemptId', request.attemptId);
|
||||
const range = normalizeRunAttemptLogReadRange(
|
||||
request.range,
|
||||
this.options.maximumReadBytes,
|
||||
);
|
||||
if (request.signal?.aborted) {
|
||||
throw new RunAttemptLogReadUnavailableError({
|
||||
cause: request.signal.reason,
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const run = await this.runs.findRunById(runId);
|
||||
if (
|
||||
!run ||
|
||||
run.id !== runId ||
|
||||
run.projectId !== projectId ||
|
||||
run.executionOwner !== 'runtime'
|
||||
) {
|
||||
return Object.freeze({ status: 'not_found' as const });
|
||||
}
|
||||
const attempt = await this.runs.findAttemptById(attemptId);
|
||||
if (
|
||||
!attempt ||
|
||||
attempt.id !== attemptId ||
|
||||
attempt.runId !== runId ||
|
||||
attempt.executorType !== this.options.executorType ||
|
||||
!RUN_ATTEMPT_STATUSES.includes(attempt.status)
|
||||
) {
|
||||
return Object.freeze({ status: 'not_found' as const });
|
||||
}
|
||||
if (attempt.logArtifactId === undefined) {
|
||||
return Object.freeze({
|
||||
status: 'pending' as const,
|
||||
projectId,
|
||||
runId,
|
||||
attemptId,
|
||||
});
|
||||
}
|
||||
if (!this.options.artifactIdPattern.test(attempt.logArtifactId)) {
|
||||
return Object.freeze({ status: 'not_found' as const });
|
||||
}
|
||||
const identity = Object.freeze({
|
||||
projectId,
|
||||
runId,
|
||||
attemptId,
|
||||
logArtifactId: attempt.logArtifactId,
|
||||
});
|
||||
const result = await this.reader.read(identity, range, request.signal);
|
||||
if (result.status === 'missing') {
|
||||
if (
|
||||
this.options.activeMissingIsPending === true &&
|
||||
!TERMINAL_ATTEMPT_STATUSES.has(attempt.status)
|
||||
) {
|
||||
return Object.freeze({ status: 'pending' as const, ...identity });
|
||||
}
|
||||
return Object.freeze({ status: 'missing' as const, ...identity });
|
||||
}
|
||||
return available(identity, range, result);
|
||||
} catch (error) {
|
||||
if (
|
||||
error instanceof InvalidRunAttemptLogReadError ||
|
||||
error instanceof RunAttemptLogReadUnavailableError
|
||||
) {
|
||||
throw error;
|
||||
}
|
||||
throw new RunAttemptLogReadUnavailableError({ cause: error });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,261 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const { test } = require('node:test');
|
||||
|
||||
const {
|
||||
InvalidRunAttemptLogReadError,
|
||||
MAX_RUN_ATTEMPT_LOG_READ_BYTES,
|
||||
RunAttemptLogReadService,
|
||||
RunAttemptLogReadUnavailableError,
|
||||
normalizeRunAttemptLogReadRange,
|
||||
} = require('../dist/run/log-read/runAttemptLogRead.js');
|
||||
|
||||
function run(overrides = {}) {
|
||||
return {
|
||||
id: 'run_123',
|
||||
projectId: 'prj_default',
|
||||
taskId: 'task_1',
|
||||
taskRevision: 'revision_1',
|
||||
triggerType: 'task_start',
|
||||
executionOrigin: 'manual',
|
||||
executionOwner: 'runtime',
|
||||
status: 'running',
|
||||
version: 2,
|
||||
eventSequence: 2,
|
||||
priority: 0,
|
||||
createdAtMs: 1,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function attempt(overrides = {}) {
|
||||
return {
|
||||
id: 'attempt_123',
|
||||
runId: 'run_123',
|
||||
attempt: 1,
|
||||
status: 'running',
|
||||
executorType: 'local_process',
|
||||
logArtifactId: `local-${'a'.repeat(30)}`,
|
||||
callbackSequence: 0,
|
||||
createdAtMs: 1,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function service(overrides = {}) {
|
||||
const calls = [];
|
||||
const runs = overrides.runs ?? {
|
||||
async findRunById() {
|
||||
return run();
|
||||
},
|
||||
async findAttemptById() {
|
||||
return attempt();
|
||||
},
|
||||
};
|
||||
const reader = overrides.reader ?? {
|
||||
async read(identity, range, signal) {
|
||||
calls.push({ identity, range, signal });
|
||||
return {
|
||||
status: 'available',
|
||||
content: Buffer.from('log'),
|
||||
start: range.offset,
|
||||
endExclusive: range.offset + 3,
|
||||
totalBytes: range.offset + 5,
|
||||
nextOffset: range.offset + 3,
|
||||
truncation: { truncated: false, maximumBytes: 1024, observedAtMs: 9 },
|
||||
};
|
||||
},
|
||||
};
|
||||
return {
|
||||
calls,
|
||||
value: new RunAttemptLogReadService(runs, reader, {
|
||||
executorType: overrides.executorType ?? 'local_process',
|
||||
artifactIdPattern: overrides.artifactIdPattern ?? /^local-[a-f0-9]{30}$/,
|
||||
maximumReadBytes: overrides.maximumReadBytes ?? 32 * 1024,
|
||||
...(overrides.activeMissingIsPending === undefined
|
||||
? {}
|
||||
: { activeMissingIsPending: overrides.activeMissingIsPending }),
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function request(overrides = {}) {
|
||||
return {
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
range: { offset: 4, length: 16 },
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test('normalizes only bounded safe ranges', () => {
|
||||
assert.deepEqual(normalizeRunAttemptLogReadRange({ offset: 0, length: 1 }), {
|
||||
offset: 0,
|
||||
length: 1,
|
||||
});
|
||||
assert.deepEqual(
|
||||
normalizeRunAttemptLogReadRange({
|
||||
offset: Number.MAX_SAFE_INTEGER,
|
||||
length: MAX_RUN_ATTEMPT_LOG_READ_BYTES,
|
||||
}),
|
||||
{ offset: Number.MAX_SAFE_INTEGER, length: MAX_RUN_ATTEMPT_LOG_READ_BYTES },
|
||||
);
|
||||
for (const range of [
|
||||
{ offset: -1, length: 1 },
|
||||
{ offset: 0.5, length: 1 },
|
||||
{ offset: 0, length: 0 },
|
||||
{ offset: 0, length: MAX_RUN_ATTEMPT_LOG_READ_BYTES + 1 },
|
||||
]) {
|
||||
assert.throws(
|
||||
() => normalizeRunAttemptLogReadRange(range),
|
||||
InvalidRunAttemptLogReadError,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('validates Project, Run, Attempt, owner and executor before storage access', async () => {
|
||||
const cases = [
|
||||
{ run: null },
|
||||
{ run: run({ projectId: 'prj_other' }) },
|
||||
{ run: run({ executionOwner: 'legacy' }) },
|
||||
{ attempt: null },
|
||||
{ attempt: attempt({ runId: 'run_other' }) },
|
||||
{ attempt: attempt({ executorType: 'remote_worker' }) },
|
||||
{ attempt: attempt({ logArtifactId: `wlog-${'a'.repeat(30)}` }) },
|
||||
];
|
||||
for (const values of cases) {
|
||||
let reads = 0;
|
||||
const { value } = service({
|
||||
runs: {
|
||||
async findRunById() {
|
||||
return values.run === undefined ? run() : values.run;
|
||||
},
|
||||
async findAttemptById() {
|
||||
return values.attempt === undefined ? attempt() : values.attempt;
|
||||
},
|
||||
},
|
||||
reader: {
|
||||
async read() {
|
||||
reads += 1;
|
||||
return { status: 'missing' };
|
||||
},
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await value.read(request()), { status: 'not_found' });
|
||||
assert.equal(reads, 0);
|
||||
}
|
||||
});
|
||||
|
||||
test('returns pending before Artifact binding and for active remote publication lag', async () => {
|
||||
const unbound = service({
|
||||
runs: {
|
||||
async findRunById() {
|
||||
return run();
|
||||
},
|
||||
async findAttemptById() {
|
||||
return attempt({ logArtifactId: undefined });
|
||||
},
|
||||
},
|
||||
});
|
||||
assert.deepEqual(await unbound.value.read(request()), {
|
||||
status: 'pending',
|
||||
projectId: 'prj_default',
|
||||
runId: 'run_123',
|
||||
attemptId: 'attempt_123',
|
||||
});
|
||||
assert.equal(unbound.calls.length, 0);
|
||||
|
||||
const remote = service({
|
||||
executorType: 'remote_worker',
|
||||
artifactIdPattern: /^wlog-[a-f0-9]{30}$/,
|
||||
activeMissingIsPending: true,
|
||||
runs: {
|
||||
async findRunById() {
|
||||
return run();
|
||||
},
|
||||
async findAttemptById() {
|
||||
return attempt({
|
||||
executorType: 'remote_worker',
|
||||
logArtifactId: `wlog-${'b'.repeat(30)}`,
|
||||
});
|
||||
},
|
||||
},
|
||||
reader: {
|
||||
async read() {
|
||||
return { status: 'missing' };
|
||||
},
|
||||
},
|
||||
});
|
||||
assert.equal((await remote.value.read(request())).status, 'pending');
|
||||
});
|
||||
|
||||
test('returns a validated bounded snapshot without copying storage bytes', async () => {
|
||||
const content = Buffer.from('log');
|
||||
const abort = new AbortController();
|
||||
const { value, calls } = service({
|
||||
reader: {
|
||||
async read(identity, range, signal) {
|
||||
assert.equal(signal, abort.signal);
|
||||
return {
|
||||
status: 'available',
|
||||
content,
|
||||
start: 4,
|
||||
endExclusive: 7,
|
||||
totalBytes: 9,
|
||||
nextOffset: 7,
|
||||
truncation: {
|
||||
truncated: true,
|
||||
maximumBytes: 64 * 1024,
|
||||
observedAtMs: 10,
|
||||
},
|
||||
};
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = await value.read(request({ signal: abort.signal }));
|
||||
assert.equal(result.status, 'available');
|
||||
assert.equal(result.content, content);
|
||||
assert.equal(result.nextOffset, 7);
|
||||
assert.deepEqual(result.truncation, {
|
||||
truncated: true,
|
||||
maximumBytes: 64 * 1024,
|
||||
observedAtMs: 10,
|
||||
});
|
||||
assert.equal(calls.length, 0);
|
||||
});
|
||||
|
||||
test('fails closed on malformed storage results and dependency failures', async () => {
|
||||
const malformed = service({
|
||||
reader: {
|
||||
async read() {
|
||||
return {
|
||||
status: 'available',
|
||||
content: Buffer.from('too-long'),
|
||||
start: 4,
|
||||
endExclusive: 12,
|
||||
totalBytes: 9,
|
||||
truncation: { truncated: 'unknown' },
|
||||
};
|
||||
},
|
||||
},
|
||||
});
|
||||
await assert.rejects(
|
||||
malformed.value.read(request()),
|
||||
RunAttemptLogReadUnavailableError,
|
||||
);
|
||||
|
||||
const failed = service({
|
||||
runs: {
|
||||
async findRunById() {
|
||||
throw new Error('database detail');
|
||||
},
|
||||
async findAttemptById() {
|
||||
return null;
|
||||
},
|
||||
},
|
||||
});
|
||||
await assert.rejects(
|
||||
failed.value.read(request()),
|
||||
RunAttemptLogReadUnavailableError,
|
||||
);
|
||||
});
|
||||
@@ -1461,6 +1461,7 @@ function auditSourceImports(root, packagePath, findings) {
|
||||
specifier.startsWith('@qinglong/local-execution/')) &&
|
||||
(packagePath !== 'packages/ql3-local-application' ||
|
||||
![
|
||||
'@qinglong/local-execution/artifact-read',
|
||||
'@qinglong/local-execution/control',
|
||||
'@qinglong/local-execution/dispatch',
|
||||
'@qinglong/local-execution/execution',
|
||||
@@ -2418,6 +2419,7 @@ function auditSourceImports(root, packagePath, findings) {
|
||||
'@qinglong/runtime-core/plugin-package-recovery',
|
||||
'@qinglong/runtime-core/plugin-package-task-publication',
|
||||
'@qinglong/runtime-core/project-tool-definition-snapshot',
|
||||
'@qinglong/runtime-core/run-attempt-log-read',
|
||||
'@qinglong/runtime-core/task-spec-semantic',
|
||||
].includes(specifier) &&
|
||||
!(
|
||||
|
||||
@@ -130,10 +130,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: localApi.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 16,
|
||||
sourceFiles: 17,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 84,
|
||||
nestedSourceFiles: 15,
|
||||
nestedSourceFiles: 16,
|
||||
rootSourceFileRoles: { 'cli.ts': 'binary_entry' },
|
||||
},
|
||||
);
|
||||
@@ -146,10 +146,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
{
|
||||
path: 'packages/ql3-local-execution',
|
||||
name: '@qinglong/local-execution',
|
||||
sourceFiles: 19,
|
||||
sourceFiles: 21,
|
||||
rootSourceFiles: 0,
|
||||
rootSourceLines: 0,
|
||||
nestedSourceFiles: 19,
|
||||
nestedSourceFiles: 21,
|
||||
rootSourceFileHardCap: 0,
|
||||
rootSourceLineHardCap: 0,
|
||||
rootSourceFileRoles: {},
|
||||
@@ -299,10 +299,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: runtimeCore.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 148,
|
||||
sourceFiles: 149,
|
||||
rootSourceFiles: 1,
|
||||
rootSourceLines: 160,
|
||||
nestedSourceFiles: 147,
|
||||
nestedSourceFiles: 148,
|
||||
rootSourceFileRoles: { 'index.ts': 'public_export' },
|
||||
},
|
||||
);
|
||||
@@ -385,10 +385,10 @@ test('current QL3 workspace has exactly eighteen reviewed package boundaries', (
|
||||
rootSourceFileRoles: clusterControl.rootSourceFileRoles,
|
||||
},
|
||||
{
|
||||
sourceFiles: 49,
|
||||
sourceFiles: 50,
|
||||
rootSourceFiles: 2,
|
||||
rootSourceLines: 195,
|
||||
nestedSourceFiles: 47,
|
||||
nestedSourceFiles: 48,
|
||||
rootSourceFileRoles: {
|
||||
'aiCli.ts': 'binary_entry',
|
||||
'cli.ts': 'binary_entry',
|
||||
|
||||
Reference in New Issue
Block a user