fix(ql3): stream sparse process logs within exact byte quotas

This commit is contained in:
whyour
2026-09-04 05:32:25 +08:00
parent ed4ba9b817
commit efb7379fa9
10 changed files with 287 additions and 12 deletions
+17
View File
@@ -716,6 +716,23 @@ jobs:
--security-opt no-new-privileges \ --security-opt no-new-privileges \
--entrypoint /usr/bin/docker \ --entrypoint /usr/bin/docker \
"${OPERATOR_IMAGE}" --version "${OPERATOR_IMAGE}" --version
- name: Verify live sparse output and byte quotas in the installed runtime
env:
IMAGE: qinglong3-local-application:ci-${{ matrix.image_arch }}
CONSOLE_IMAGE: qinglong3-local-console:ci-${{ matrix.image_arch }}
run: |
for capture_image in "${IMAGE}" "${CONSOLE_IMAGE}"; do
docker run --rm --read-only \
--network none \
--cap-drop ALL \
--security-opt no-new-privileges \
--memory=128m --memory-swap=128m --cpus=0.5 --pids-limit=64 \
--tmpfs /tmp:rw,nosuid,nodev,noexec,size=16m \
--volume "${GITHUB_WORKSPACE}/packages/ql3-local-process/test:/opt/qinglong/node_modules/@qinglong/local-process/test:ro" \
--entrypoint node "${capture_image}" \
--test --test-name-pattern='publishes sparse|keeps byte-exact|rejects unsupported' \
/opt/qinglong/node_modules/@qinglong/local-process/test/launcher.test.cjs
done
- name: Build the short-lived fresh deployment authority - name: Build the short-lived fresh deployment authority
working-directory: packages/ql3-local-owner-cli working-directory: packages/ql3-local-owner-cli
run: node ../../scripts/ql3-build-package-closure.cjs run: node ../../scripts/ql3-build-package-closure.cjs
@@ -1,6 +1,6 @@
# ADR-0535:现有面板的规范 Task/Run 执行管理 # ADR-0535:现有面板的规范 Task/Run 执行管理
- 状态:AcceptedD-433 原始源码主 CI 已通过;客户端组合门与实物待验证) - 状态:AcceptedD-433 原始源码主 CI 已通过;增强组合门发现运行中日志缺陷,修复待 Linux 验证)
- 日期:2026-09-04 - 日期:2026-09-04
- 关联:QL-RFC-0001 D-433、ADR-0530、ADR-0531、ADR-0534 - 关联:QL-RFC-0001 D-433、ADR-0530、ADR-0531、ADR-0534
@@ -29,7 +29,7 @@ Node 20 legacy migration toolchain 的 production panel build 通过;裁剪包
本切片不是完整旧面板迁移:任务/定时创建编辑、脚本、订阅、环境变量、依赖与多用户远程 Web 会话仍未闭合。Local 仍仅 loopback/SSH tunnel;不可将本地 mock 或 unit tests 当成实际 Owner 写入、公开发布或双架构镜像证据。D-431 实物不含本切片,交付必须绑定自己的 source revision。 本切片不是完整旧面板迁移:任务/定时创建编辑、脚本、订阅、环境变量、依赖与多用户远程 Web 会话仍未闭合。Local 仍仅 loopback/SSH tunnel;不可将本地 mock 或 unit tests 当成实际 Owner 写入、公开发布或双架构镜像证据。D-431 实物不含本切片,交付必须绑定自己的 source revision。
## 真实客户端组合验收(待 Linux 执行) ## 真实客户端组合验收
已有 `ql3-local-api-cancellation-live-contract` 不再手写面板启动与取消请求,而是在宿主机将实际 `src/utils/qinglong3.ts``src/components/qinglong3/runControl.ts` 编译为临时测试模块,在同一个隔离 Linux 网络命名空间中使用真实 fetch 连接生产 Local API。记录两个源文件的 SHA-256TypeScript、VM 适配器与生成模块均不进入产品制品,不新增 workspace package 或生产依赖。 已有 `ql3-local-api-cancellation-live-contract` 不再手写面板启动与取消请求,而是在宿主机将实际 `src/utils/qinglong3.ts``src/components/qinglong3/runControl.ts` 编译为临时测试模块,在同一个隔离 Linux 网络命名空间中使用真实 fetch 连接生产 Local API。记录两个源文件的 SHA-256TypeScript、VM 适配器与生成模块均不进入产品制品,不新增 workspace package 或生产依赖。
@@ -56,3 +56,24 @@ Node 20 legacy migration toolchain 的 production panel build 通过;裁剪包
两架构最终 Application layer 的原生 `console.js` 均为 82,521 bytesSHA-256 `dfcba011a165743052fe290e4f4c6b6b90074fde8dec20392252100332a6eaed`,与该 source revision 逐字节相同,包含会话隔离修复;Crontab chunk 为 76,500 bytesSHA-256 `16f7f79622947024b84c7bd43c74b81841a2372d90a09864e473fda863794446`,核对含 `task_run_v1`、版本确认、规范 cancellation 与 receipt 校验代码。240 文件 panel manifest 的两架构摘要同为 `923ea32ae7c08624f495e7e90aa9b37a66edfc69bbda3d971001d06cb439e842` 两架构最终 Application layer 的原生 `console.js` 均为 82,521 bytesSHA-256 `dfcba011a165743052fe290e4f4c6b6b90074fde8dec20392252100332a6eaed`,与该 source revision 逐字节相同,包含会话隔离修复;Crontab chunk 为 76,500 bytesSHA-256 `16f7f79622947024b84c7bd43c74b81841a2372d90a09864e473fda863794446`,核对含 `task_run_v1`、版本确认、规范 cancellation 与 receipt 校验代码。240 文件 panel manifest 的两架构摘要同为 `923ea32ae7c08624f495e7e90aa9b37a66edfc69bbda3d971001d06cb439e842`
这些证据只排除了“包里仍是旧页面”的问题,没有补齐浏览器端到端、客户端 Linux 组合门或成功 milestone;失败流水线的中间包不提供给用户部署,不替代 D-431 已交付阶段包。 这些证据只排除了“包里仍是旧页面”的问题,没有补齐浏览器端到端、客户端 Linux 组合门或成功 milestone;失败流水线的中间包不提供给用户部署,不替代 D-431 已交付阶段包。
## 运行中稀疏日志缺陷与修复
提交 `ed4ba9b817b859ca1afe8b0552116b86d90f92c7` 的主 CI [33805484737](https://github.com/whyour/qinglong/actions/runs/33805484737) 已失败。amd64 job `100816011201` 与 arm64 job `100816011278` 均在真实客户端 Edge 组合门报告 `panel reads running process log marker did not converge`,尚未进入随后取消与 Standalone 验收;此前已完成的 Console fresh Owner/终态日志检查不能替代运行中日志证明。
生产 `local-process` 启动器的 FIFO 截流器原用 `head -c`;小段输出在进程持续运行时留在工具的 stdio 缓冲区。新增回归让真实子进程写出包含 NUL/0xff、无换行的 5 字节标记,等待测试端确认实际日志文件可读后才释放退出;argv/shell 两条路径在修复前均失败(文件仍为 0 字节),不会靠增大输出或先结束进程掩盖问题。
修复仅替换已打包的 3.0 启动器采集原语并更新其审核摘要:
- Alpine/BusyBox:显式调用 `busybox dd bs=16384 count=remaining iflag=count_bytes status=none`,每次短读直接写出,按真实字节扣减余额。
- Debian/GNU:使用 `stdbuf -o0 head -c remaining`,保留 GNU head 的精确读取边界,关闭其输出缓冲。
- 启动用户代码前验证所需工具能力;不支持则退出 125,不回退到缓冲复制或逐字节系统调用。工具错误不得通过继承的 stderr 绕过日志限额;复制失败仍排空用户输出,但不发布虚假的完整截断事实,读取侧保持 unknown。
- 不增加 package、OS 安装项、每任务 Node sidecar、监听器、timer 或持久状态;两种工具组合来自现有 Linux 基础环境。`shell/ql3-launcher.sh` 属遗留路径,本次不修改。
曾评估统一使用 GNU/BusyBox `dd iflag=count_bytes`,但 GNU dd 的短读计数使原有 64 KiB 精确配额回归只保留 32 KiB,因此该候选已撤回;`fullblock` 又会重新引入等待完整块的问题。不能将两种实现当作等价。
回归还覆盖空输出、恰好上限、超出一个字节、已有日志的非整块剩余额度、余额为零有/无新输出,以及工具不兼容和复制故障。Local 与 Worker 共用启动器,因此 Worker POSIX 测试同步使用真实 GNU 工具验证;macOS 测试只替换工具查找目录并保留既有 fake identity,不声称提供 macOS 生产进程身份支持。
主 CI 的 amd64/arm64 Local image job 增加安装后 headless/Console 镜像的稀疏输出与精确字节测试,保持 128 MiB、0.5 CPU、64 PID、只读、无网络及 noexec tmpfs 条件。该测试读取镜像自身 dist/assets,只把测试代码只读挂入;原实际面板客户端 Linux 组合门及其稳定步骤名不变。修复需由新提交的 Linux CI 和后续成功产物流水线证明,不把本地结果或旧归档算作已交付。
本地最终回归:Local process + Worker 163/163(首次受限环境的 3 项 loopback EPERM 已在获准环境重跑通过);完整后端 1705 项,1703 pass / 2 条件 skip / 0 fail;镜像/客户端专项 24/24Worker 编译与类型检查、Local/Operator image audit、18-package boundary audit、YAML 解析与新增步骤 shell 语法检查通过。本机 Docker Engine 仍不可连接,没有本地 Linux/Alpine 成功证据。
+1 -1
View File
@@ -29,7 +29,7 @@ D-431 给原生 Task、Trigger、Run 列表增加页脚“下一页/回到首
D-433 源码候选为旧面板定时行增加“执行管理”:先展示当前 Task revision,再显式确认运行;选择确切 Run 后可以刷新、读该次日志或请求取消。它不恢复旧 Cron 写接口,也不修改定时绑定版本。网络结果不明时只重试同一 mutation,关闭窗口不会撤销已提交请求。此功能需服务端 capability 显式开放,不包含在上表任何旧归档中,见 [ADR-0535](../adr/ADR-0535-canonical-panel-run-control.md)。 D-433 源码候选为旧面板定时行增加“执行管理”:先展示当前 Task revision,再显式确认运行;选择确切 Run 后可以刷新、读该次日志或请求取消。它不恢复旧 Cron 写接口,也不修改定时绑定版本。网络结果不明时只重试同一 mutation,关闭窗口不会撤销已提交请求。此功能需服务端 capability 显式开放,不包含在上表任何旧归档中,见 [ADR-0535](../adr/ADR-0535-canonical-panel-run-control.md)。
执行管理的后续验收已接入实际面板客户端与 Linux Local API 组合门,要求读取真实进程日志、取消响应丢失后的精确重试以及重启后持久状态/日志;当前仍待 Linux 实跑。该门使用隔离数据库夹具,不是浏览器与 Owner 初始化全链路证明。阶段包可用与“原版 2.x 所有页面可直接替换后端”是不同结论,请按上表的 source revision 选择。 执行管理的后续验收已接入实际面板客户端与 Linux Local API 组合门,要求读取真实进程日志、取消响应丢失后的精确重试以及重启后持久状态/日志`ed4ba9b` 双架构实跑发现运行中稀疏日志不能及时读到,启动器修复及镜像回归尚待新 Linux CI 验证,不能将该候选视为可交付。该门使用隔离数据库夹具,不是浏览器与 Owner 初始化全链路证明。阶段包可用与“原版 2.x 所有页面可直接替换后端”是不同结论,请按上表的 source revision 选择。
D-426b2c 又补齐了 Console 镜像的 adopted-target 入口证据:切换演练使用 `ql3-local-api --cutover-probe --config <local-api.json>`,同时绑定外层 API 配置、内层 Application 配置与 exact mounts,但该模式不会启动本页使用的 listener、credential、scheduler 或 mutation surface。正常启动仍使用下文不带 `--cutover-probe` 的命令;提交 `229c3cb4e826866a0c7c4d81cb5e52cdc3975eec` 的 [artifact run 33463415938](https://github.com/whyour/qinglong/actions/runs/33463415938) 已交付 exact amd64/arm64 Console Trial Kit 与 milestone,三份下载产物的离线 auditor 均为 `compatible=true` D-426b2c 又补齐了 Console 镜像的 adopted-target 入口证据:切换演练使用 `ql3-local-api --cutover-probe --config <local-api.json>`,同时绑定外层 API 配置、内层 Application 配置与 exact mounts,但该模式不会启动本页使用的 listener、credential、scheduler 或 mutation surface。正常启动仍使用下文不带 `--cutover-probe` 的命令;提交 `229c3cb4e826866a0c7c4d81cb5e52cdc3975eec` 的 [artifact run 33463415938](https://github.com/whyour/qinglong/actions/runs/33463415938) 已交付 exact amd64/arm64 Console Trial Kit 与 milestone,三份下载产物的离线 auditor 均为 `compatible=true`
@@ -63,6 +63,19 @@ if [ -n "$output_quota_fifo" ]; then
[ -z "$output_truncation_temporary" ]; then [ -z "$output_truncation_temporary" ]; then
exit 125 exit 125
fi fi
# Use each supported Linux userspace's byte-exact, streaming copy primitive.
# GNU dd counts short reads as blocks even with count_bytes; do not use it.
# BusyBox head can read ahead past the quota; do not use it either.
if command -v busybox >/dev/null 2>&1 &&
busybox dd bs=16384 count=0 iflag=count_bytes status=none </dev/null >/dev/null 2>&1; then
output_copy=busybox
elif head --version >/dev/null 2>&1 &&
stdbuf -o0 head -c 0 </dev/null >/dev/null 2>&1; then
output_copy=gnu
else
# No buffered or per-byte fallback on unsupported systems.
exit 125
fi
rm -f "$output_truncation_temporary" 2>/dev/null || exit 125 rm -f "$output_truncation_temporary" 2>/dev/null || exit 125
if ! mkfifo -m 600 "$output_quota_fifo" 2>/dev/null; then if ! mkfifo -m 600 "$output_quota_fifo" 2>/dev/null; then
exit 125 exit 125
@@ -83,15 +96,25 @@ if [ -n "$output_quota_fifo" ]; then
} }
( (
capture_succeeded=true
if [ "$output_quota_remaining_bytes" -gt 0 ]; then if [ "$output_quota_remaining_bytes" -gt 0 ]; then
head -c "$output_quota_remaining_bytes" # Diagnostics must not bypass that quota through inherited stderr.
if [ "$output_copy" = busybox ]; then
busybox dd bs=16384 count="$output_quota_remaining_bytes" iflag=count_bytes status=none 2>/dev/null || capture_succeeded=false
else
stdbuf -o0 head -c "$output_quota_remaining_bytes" 2>/dev/null || capture_succeeded=false
fi
fi fi
overflow_bytes=$(wc -c | tr -d '[:space:]') || overflow_bytes= overflow_bytes=$(wc -c | tr -d '[:space:]') || overflow_bytes=
case "$overflow_bytes" in # A failed copy cannot attest complete capture. Still drain the producer;
''|*[!0-9]*) ;; # leave truncation unknown and retain its own exit/receipt semantics.
0) publish_output_truncation false ;; if [ "$capture_succeeded" = true ]; then
*) publish_output_truncation true ;; case "$overflow_bytes" in
esac ''|*[!0-9]*) ;;
0) publish_output_truncation false ;;
*) publish_output_truncation true ;;
esac
fi
) < "$output_quota_fifo" & ) < "$output_quota_fifo" &
drain_pid=$! drain_pid=$!
fi fi
@@ -12,7 +12,7 @@ import {
} from './localProcessIdentity'; } from './localProcessIdentity';
export const BUNDLED_LOCAL_PROCESS_LAUNCHER_SHA256 = export const BUNDLED_LOCAL_PROCESS_LAUNCHER_SHA256 =
'db4342ea57f8f7f19e385e204889ac03e97a2f82f42b01f2e59291be4b569153'; '96653ca2b788f9a85fc3313fdf1b33b3482c121fd92b6344b26510e48a1d9804';
export const MAX_LOCAL_PROCESS_ENVIRONMENT_ENTRIES = 256; export const MAX_LOCAL_PROCESS_ENVIRONMENT_ENTRIES = 256;
export const MAX_LOCAL_PROCESS_ENVIRONMENT_BYTES = 64 * 1024; export const MAX_LOCAL_PROCESS_ENVIRONMENT_BYTES = 64 * 1024;
export const MAX_LOCAL_PROCESS_ARGUMENTS = 256; export const MAX_LOCAL_PROCESS_ARGUMENTS = 256;
@@ -0,0 +1,22 @@
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { spawnSync } = require('node:child_process');
// macOS has no production /proc identity. Tests with a fake identity provider
// still use real GNU utilities, so BSD semantics cannot mask Linux regressions.
function quotaEnvironment(directory) {
if (process.platform !== 'darwin') return {};
const bin = path.join(directory, 'quota-bin');
fs.mkdirSync(bin, { mode: 0o700 });
for (const [name, installed] of [['head', 'ghead'], ['stdbuf', 'stdbuf']]) {
const found = spawnSync('/bin/sh', ['-c', `command -v ${installed}`], {
encoding: 'utf8',
});
assert.equal(found.status, 0, `macOS launcher tests require coreutils ${installed}`);
fs.symlinkSync(found.stdout.trim(), path.join(bin, name));
}
return { PATH: `${bin}:/usr/bin:/bin` };
}
module.exports = { quotaEnvironment };
@@ -2,6 +2,7 @@ const assert = require('node:assert/strict');
const fs = require('node:fs'); const fs = require('node:fs');
const os = require('node:os'); const os = require('node:os');
const path = require('node:path'); const path = require('node:path');
const { quotaEnvironment } = require('./helpers/quotaEnvironment.cjs');
const { test } = require('node:test'); const { test } = require('node:test');
const { CompletionReceiptFileStore } = require('../dist'); const { CompletionReceiptFileStore } = require('../dist');
const { LocalProcessLaunchError, LocalProcessLauncher } = require('../dist'); const { LocalProcessLaunchError, LocalProcessLauncher } = require('../dist');
@@ -212,6 +213,7 @@ test('hard-caps durable output and publishes an immutable truncation fact', asyn
attemptId: ATTEMPT_ID, attemptId: ATTEMPT_ID,
callbackSequence: 1, callbackSequence: 1,
callbackToken: TOKEN, callbackToken: TOKEN,
environment: quotaEnvironment(directory),
command: { command: {
kind: 'argv', kind: 'argv',
file: process.execPath, file: process.execPath,
@@ -261,3 +263,162 @@ test('hard-caps durable output and publishes an immutable truncation fact', asyn
[], [],
); );
}); });
for (const kind of ['argv', 'shell']) {
test(`publishes sparse binary ${kind} output before the process exits`, async (t) => {
const { directory, receiptRoot } = fixture(t);
const logArtifactId = `local-${'c'.repeat(30)}`;
const outputFilePath = path.join(directory, `${logArtifactId}.log`);
const release = path.join(directory, 'release');
const marker = Buffer.from([0x00, 0xff, 0x71, 0x6c, 0x33]);
const script = path.join(directory, 'producer.cjs');
fs.writeFileSync(script, `
const fs = require('node:fs');
process.stdout.write(Buffer.from([0x00, 0xff, 0x71, 0x6c, 0x33]));
const deadline = setTimeout(() => process.exit(92), 15000);
const timer = setInterval(() => {
if (!fs.existsSync(process.argv[2])) return;
clearInterval(timer);
clearTimeout(deadline);
process.stderr.write('tail', () => process.exit(7));
}, 10);
`, { mode: 0o600 });
const launcher = new LocalProcessLauncher(
{ register: async () => undefined },
{ receiptRoot, identityProvider: identityProvider() },
);
const quote = (value) => `'${value.replaceAll("'", "'\\''")}'`;
const handle = await launcher.start({
runId: RUN_ID, attemptId: ATTEMPT_ID,
callbackSequence: 1, callbackToken: TOKEN,
environment: quotaEnvironment(directory),
command: kind === 'argv'
? { kind, file: process.execPath, args: [script, release] }
: { kind, command: [process.execPath, script, release].map(quote).join(' ') },
output: { filePath: outputFilePath, maximumBytes: 65536, logArtifactId },
});
try {
const deadline = Date.now() + 5000;
while (fs.statSync(outputFilePath).size < marker.length && Date.now() < deadline) {
await new Promise((resolve) => setTimeout(resolve, 10));
}
assert.deepEqual(fs.readFileSync(outputFilePath), marker,
'sparse bytes must be readable while the producer is waiting for release');
assert.equal(await new CompletionReceiptFileStore(receiptRoot).read(ATTEMPT_ID), undefined);
fs.writeFileSync(release, '', { mode: 0o600 });
assert.deepEqual(await handle.completion, { exitCode: 7, signal: null });
assert.deepEqual(fs.readFileSync(outputFilePath), Buffer.concat([marker, Buffer.from('tail')]));
const fact = JSON.parse(fs.readFileSync(path.join(directory, `.${logArtifactId}.log.truncated.json`)));
assert.equal(fact.quotaReached, false);
assert.equal((await waitForReceipt(new CompletionReceiptFileStore(receiptRoot))).exitCode, 7);
} finally {
fs.writeFileSync(release, '', { mode: 0o600 });
await handle.completion;
}
});
}
for (const [label, initialBytes, producedBytes] of [
['empty output', 0, 0],
['exact quota', 0, 65536],
['one-byte overflow', 0, 65537],
['partial final block', 65513, 31],
['exhausted quota', 65536, 23],
['exhausted quota without overflow', 65536, 0],
]) {
test(`keeps byte-exact capture and truncation for ${label}`, async (t) => {
const { directory, receiptRoot } = fixture(t);
const logArtifactId = `local-${'d'.repeat(30)}`;
const filePath = path.join(directory, `${logArtifactId}.log`);
const initial = Buffer.alloc(initialBytes, 0x5a);
fs.writeFileSync(filePath, initial, { mode: 0o600 });
const produced = Buffer.from(Array.from({ length: producedBytes }, (_, i) => i % 256));
const launcher = new LocalProcessLauncher(
{ register: async () => undefined },
{ receiptRoot, identityProvider: identityProvider() },
);
const handle = await launcher.start({
runId: RUN_ID, attemptId: ATTEMPT_ID,
callbackSequence: 1, callbackToken: TOKEN,
environment: quotaEnvironment(directory),
command: { kind: 'argv', file: process.execPath, args: ['-e', `
const data = Buffer.from(Array.from({length: ${producedBytes}}, (_, i) => i % 256));
let offset = 0;
function write() {
if (offset === data.length) return process.exit(9);
const end = Math.min(data.length, offset + 997);
const chunk = data.subarray(offset, end);
offset = end;
process.stdout.write(chunk, () => setImmediate(write));
}
write();
`] },
output: { filePath, maximumBytes: 65536, logArtifactId },
});
assert.deepEqual(await handle.completion, { exitCode: 9, signal: null });
assert.deepEqual(fs.readFileSync(filePath), Buffer.concat([initial, produced]).subarray(0, 65536));
const fact = JSON.parse(fs.readFileSync(path.join(directory, `.${logArtifactId}.log.truncated.json`)));
assert.equal(fact.quotaReached, initialBytes + producedBytes > 65536);
assert.equal((await waitForReceipt(new CompletionReceiptFileStore(receiptRoot))).exitCode, 9);
});
}
test('rejects unsupported capture utilities before running user code', async (t) => {
const { directory, receiptRoot } = fixture(t);
const bin = path.join(directory, 'unsupported-bin');
fs.mkdirSync(bin, { mode: 0o700 });
for (const name of ['busybox', 'head', 'stdbuf']) {
fs.writeFileSync(path.join(bin, name), '#!/bin/sh\nexit 1\n', { mode: 0o700 });
}
const marker = path.join(directory, 'must-not-run');
const logArtifactId = `local-${'e'.repeat(30)}`;
const filePath = path.join(directory, `${logArtifactId}.log`);
const launcher = new LocalProcessLauncher(
{ register: async () => undefined },
{ receiptRoot, identityProvider: identityProvider() },
);
const handle = await launcher.start({
runId: RUN_ID, attemptId: ATTEMPT_ID,
callbackSequence: 1, callbackToken: TOKEN,
environment: { PATH: `${bin}:/usr/bin:/bin` },
command: { kind: 'argv', file: '/usr/bin/touch', args: [marker] },
output: { filePath, maximumBytes: 65536, logArtifactId },
});
assert.deepEqual(await handle.completion, { exitCode: 125, signal: null });
assert.equal(fs.existsSync(marker), false);
assert.equal(fs.statSync(filePath).size, 0);
assert.equal(await new CompletionReceiptFileStore(receiptRoot).read(ATTEMPT_ID), undefined);
assert.equal(fs.readdirSync(directory).some((name) => name.endsWith('.fifo')), false);
});
test('capture failure drains output without forging a truncation fact or changing user exit', async (t) => {
const { directory, receiptRoot } = fixture(t);
const bin = path.join(directory, 'failing-bin');
fs.mkdirSync(bin, { mode: 0o700 });
fs.writeFileSync(path.join(bin, 'busybox'), `#!/bin/sh
case " $* " in
*' count=0 '*) exit 0 ;;
esac
printf 'capture failure must not bypass the log quota' >&2
exit 1
`, { mode: 0o700 });
const logArtifactId = `local-${'f'.repeat(30)}`;
const filePath = path.join(directory, `${logArtifactId}.log`);
const launcher = new LocalProcessLauncher(
{ register: async () => undefined },
{ receiptRoot, identityProvider: identityProvider() },
);
const handle = await launcher.start({
runId: RUN_ID, attemptId: ATTEMPT_ID,
callbackSequence: 1, callbackToken: TOKEN,
environment: { PATH: `${bin}:/usr/bin:/bin` },
command: { kind: 'argv', file: process.execPath, args: [
'-e', 'process.stdout.write(Buffer.alloc(256 * 1024), () => process.exit(9));',
] },
output: { filePath, maximumBytes: 65536, logArtifactId },
});
assert.deepEqual(await handle.completion, { exitCode: 9, signal: null });
assert.equal(fs.statSync(filePath).size, 0);
assert.equal(fs.existsSync(path.join(directory, `.${logArtifactId}.log.truncated.json`)), false);
assert.equal((await waitForReceipt(new CompletionReceiptFileStore(receiptRoot))).exitCode, 9);
});
@@ -4,7 +4,8 @@ const assert = require('node:assert/strict');
const fs = require('node:fs/promises'); const fs = require('node:fs/promises');
const os = require('node:os'); const os = require('node:os');
const path = require('node:path'); const path = require('node:path');
const { test } = require('node:test'); const { test, before, after } = require('node:test');
const { quotaEnvironment } = require('../../ql3-local-process/test/helpers/quotaEnvironment.cjs');
const { CompletionReceiptFileStore } = require('@qinglong/local-process'); const { CompletionReceiptFileStore } = require('@qinglong/local-process');
const { const {
WorkerFileLogArtifactAllocator, WorkerFileLogArtifactAllocator,
@@ -17,6 +18,16 @@ const {
const RUN_ID = '019f70e0-0000-7000-8000-000000000101'; const RUN_ID = '019f70e0-0000-7000-8000-000000000101';
const ATTEMPT_ID = '019f70e0-0000-7000-8000-000000000102'; const ATTEMPT_ID = '019f70e0-0000-7000-8000-000000000102';
const TOKEN = Buffer.alloc(32, 0x5a); const TOKEN = Buffer.alloc(32, 0x5a);
let utilityRoot;
let utilityEnvironment = [];
before(async () => {
utilityRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'ql3-worker-utilities-'));
utilityEnvironment = Object.entries(quotaEnvironment(utilityRoot))
.map(([name, value]) => ({ name, value }));
});
after(async () => {
if (utilityRoot) await fs.rm(utilityRoot, { recursive: true, force: true });
});
async function fixture(t) { async function fixture(t) {
const root = await fs.mkdtemp(path.join(os.tmpdir(), 'ql3-worker-posix-')); const root = await fs.mkdtemp(path.join(os.tmpdir(), 'ql3-worker-posix-'));
@@ -78,7 +89,7 @@ function launch(prepared, output, overrides = {}) {
"process.stdout.write(process.env.QL3_RECEIPT_CALLBACK_TOKEN ? 'leaked' : 'worker-output')", "process.stdout.write(process.env.QL3_RECEIPT_CALLBACK_TOKEN ? 'leaked' : 'worker-output')",
], ],
}, },
environment: [], environment: utilityEnvironment,
logArtifactId: prepared.logArtifactId, logArtifactId: prepared.logArtifactId,
output, output,
completionCallback: { sequence: 1, token: Buffer.from(TOKEN) }, completionCallback: { sequence: 1, token: Buffer.from(TOKEN) },
+4
View File
@@ -442,6 +442,10 @@ function auditWorkflow(contents, findings) {
'--security-opt no-new-privileges', '--security-opt no-new-privileges',
'--memory=128m', '--memory=128m',
'--pids-limit=64', '--pids-limit=64',
'Verify live sparse output and byte quotas in the installed runtime',
'for capture_image in "${IMAGE}" "${CONSOLE_IMAGE}"; do',
'--test --test-name-pattern=\'publishes sparse|keeps byte-exact|rejects unsupported\'',
'/opt/qinglong/node_modules/@qinglong/local-process/test/launcher.test.cjs',
'scripts/ql3-local-image-inventory.cjs', 'scripts/ql3-local-image-inventory.cjs',
'scripts/ql3-local-console-image-inventory.cjs', 'scripts/ql3-local-console-image-inventory.cjs',
'--inventory-root=/opt/qinglong/node_modules', '--inventory-root=/opt/qinglong/node_modules',
+16
View File
@@ -137,6 +137,22 @@ test('rejects a Local runtime without the verified process launcher asset', () =
} }
}); });
test('rejects removal of the installed runtime sparse output gate', () => {
const current = fixture();
try {
const workflow = path.join(current.root, '.github/workflows/ql3-ci.yml');
fs.writeFileSync(workflow, fs.readFileSync(workflow, 'utf8').replace(
'--test --test-name-pattern=\'publishes sparse|keeps byte-exact|rejects unsupported\'',
'--test --test-name-pattern=\'no-tests\'',
));
const report = auditLocalImageContract(current.root);
assert.equal(report.compatible, false);
assert.ok(report.findings.some(({ code }) => code === 'LOCAL_IMAGE_CI_CONTRACT_DRIFT'));
} finally {
current.close();
}
});
test('rejects a mutable runtime base image', () => { test('rejects a mutable runtime base image', () => {
const current = fixture(); const current = fixture();
try { try {