mirror of
https://github.com/whyour/qinglong.git
synced 2026-09-20 08:05:22 +08:00
fix(ql3): stream sparse process logs within exact byte quotas
This commit is contained in:
@@ -716,6 +716,23 @@ jobs:
|
||||
--security-opt no-new-privileges \
|
||||
--entrypoint /usr/bin/docker \
|
||||
"${OPERATOR_IMAGE}" --version
|
||||
- name: Verify live sparse output and byte quotas in the installed runtime
|
||||
env:
|
||||
IMAGE: qinglong3-local-application:ci-${{ matrix.image_arch }}
|
||||
CONSOLE_IMAGE: qinglong3-local-console:ci-${{ matrix.image_arch }}
|
||||
run: |
|
||||
for capture_image in "${IMAGE}" "${CONSOLE_IMAGE}"; do
|
||||
docker run --rm --read-only \
|
||||
--network none \
|
||||
--cap-drop ALL \
|
||||
--security-opt no-new-privileges \
|
||||
--memory=128m --memory-swap=128m --cpus=0.5 --pids-limit=64 \
|
||||
--tmpfs /tmp:rw,nosuid,nodev,noexec,size=16m \
|
||||
--volume "${GITHUB_WORKSPACE}/packages/ql3-local-process/test:/opt/qinglong/node_modules/@qinglong/local-process/test:ro" \
|
||||
--entrypoint node "${capture_image}" \
|
||||
--test --test-name-pattern='publishes sparse|keeps byte-exact|rejects unsupported' \
|
||||
/opt/qinglong/node_modules/@qinglong/local-process/test/launcher.test.cjs
|
||||
done
|
||||
- name: Build the short-lived fresh deployment authority
|
||||
working-directory: packages/ql3-local-owner-cli
|
||||
run: node ../../scripts/ql3-build-package-closure.cjs
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# ADR-0535:现有面板的规范 Task/Run 执行管理
|
||||
|
||||
- 状态:Accepted(D-433 原始源码主 CI 已通过;客户端组合门与实物待验证)
|
||||
- 状态:Accepted(D-433 原始源码主 CI 已通过;增强组合门发现运行中日志缺陷,修复待 Linux 验证)
|
||||
- 日期:2026-09-04
|
||||
- 关联:QL-RFC-0001 D-433、ADR-0530、ADR-0531、ADR-0534
|
||||
|
||||
@@ -29,7 +29,7 @@ Node 20 legacy migration toolchain 的 production panel build 通过;裁剪包
|
||||
|
||||
本切片不是完整旧面板迁移:任务/定时创建编辑、脚本、订阅、环境变量、依赖与多用户远程 Web 会话仍未闭合。Local 仍仅 loopback/SSH tunnel;不可将本地 mock 或 unit tests 当成实际 Owner 写入、公开发布或双架构镜像证据。D-431 实物不含本切片,交付必须绑定自己的 source revision。
|
||||
|
||||
## 真实客户端组合验收(待 Linux 执行)
|
||||
## 真实客户端组合验收
|
||||
|
||||
已有 `ql3-local-api-cancellation-live-contract` 不再手写面板启动与取消请求,而是在宿主机将实际 `src/utils/qinglong3.ts` 和 `src/components/qinglong3/runControl.ts` 编译为临时测试模块,在同一个隔离 Linux 网络命名空间中使用真实 fetch 连接生产 Local API。记录两个源文件的 SHA-256;TypeScript、VM 适配器与生成模块均不进入产品制品,不新增 workspace package 或生产依赖。
|
||||
|
||||
@@ -56,3 +56,24 @@ Node 20 legacy migration toolchain 的 production panel build 通过;裁剪包
|
||||
两架构最终 Application layer 的原生 `console.js` 均为 82,521 bytes,SHA-256 `dfcba011a165743052fe290e4f4c6b6b90074fde8dec20392252100332a6eaed`,与该 source revision 逐字节相同,包含会话隔离修复;Crontab chunk 为 76,500 bytes,SHA-256 `16f7f79622947024b84c7bd43c74b81841a2372d90a09864e473fda863794446`,核对含 `task_run_v1`、版本确认、规范 cancellation 与 receipt 校验代码。240 文件 panel manifest 的两架构摘要同为 `923ea32ae7c08624f495e7e90aa9b37a66edfc69bbda3d971001d06cb439e842`。
|
||||
|
||||
这些证据只排除了“包里仍是旧页面”的问题,没有补齐浏览器端到端、客户端 Linux 组合门或成功 milestone;失败流水线的中间包不提供给用户部署,不替代 D-431 已交付阶段包。
|
||||
|
||||
## 运行中稀疏日志缺陷与修复
|
||||
|
||||
提交 `ed4ba9b817b859ca1afe8b0552116b86d90f92c7` 的主 CI [33805484737](https://github.com/whyour/qinglong/actions/runs/33805484737) 已失败。amd64 job `100816011201` 与 arm64 job `100816011278` 均在真实客户端 Edge 组合门报告 `panel reads running process log marker did not converge`,尚未进入随后取消与 Standalone 验收;此前已完成的 Console fresh Owner/终态日志检查不能替代运行中日志证明。
|
||||
|
||||
生产 `local-process` 启动器的 FIFO 截流器原用 `head -c`;小段输出在进程持续运行时留在工具的 stdio 缓冲区。新增回归让真实子进程写出包含 NUL/0xff、无换行的 5 字节标记,等待测试端确认实际日志文件可读后才释放退出;argv/shell 两条路径在修复前均失败(文件仍为 0 字节),不会靠增大输出或先结束进程掩盖问题。
|
||||
|
||||
修复仅替换已打包的 3.0 启动器采集原语并更新其审核摘要:
|
||||
|
||||
- Alpine/BusyBox:显式调用 `busybox dd bs=16384 count=remaining iflag=count_bytes status=none`,每次短读直接写出,按真实字节扣减余额。
|
||||
- Debian/GNU:使用 `stdbuf -o0 head -c remaining`,保留 GNU head 的精确读取边界,关闭其输出缓冲。
|
||||
- 启动用户代码前验证所需工具能力;不支持则退出 125,不回退到缓冲复制或逐字节系统调用。工具错误不得通过继承的 stderr 绕过日志限额;复制失败仍排空用户输出,但不发布虚假的完整截断事实,读取侧保持 unknown。
|
||||
- 不增加 package、OS 安装项、每任务 Node sidecar、监听器、timer 或持久状态;两种工具组合来自现有 Linux 基础环境。`shell/ql3-launcher.sh` 属遗留路径,本次不修改。
|
||||
|
||||
曾评估统一使用 GNU/BusyBox `dd iflag=count_bytes`,但 GNU dd 的短读计数使原有 64 KiB 精确配额回归只保留 32 KiB,因此该候选已撤回;`fullblock` 又会重新引入等待完整块的问题。不能将两种实现当作等价。
|
||||
|
||||
回归还覆盖空输出、恰好上限、超出一个字节、已有日志的非整块剩余额度、余额为零有/无新输出,以及工具不兼容和复制故障。Local 与 Worker 共用启动器,因此 Worker POSIX 测试同步使用真实 GNU 工具验证;macOS 测试只替换工具查找目录并保留既有 fake identity,不声称提供 macOS 生产进程身份支持。
|
||||
|
||||
主 CI 的 amd64/arm64 Local image job 增加安装后 headless/Console 镜像的稀疏输出与精确字节测试,保持 128 MiB、0.5 CPU、64 PID、只读、无网络及 noexec tmpfs 条件。该测试读取镜像自身 dist/assets,只把测试代码只读挂入;原实际面板客户端 Linux 组合门及其稳定步骤名不变。修复需由新提交的 Linux CI 和后续成功产物流水线证明,不把本地结果或旧归档算作已交付。
|
||||
|
||||
本地最终回归:Local process + Worker 163/163(首次受限环境的 3 项 loopback EPERM 已在获准环境重跑通过);完整后端 1705 项,1703 pass / 2 条件 skip / 0 fail;镜像/客户端专项 24/24,Worker 编译与类型检查、Local/Operator image audit、18-package boundary audit、YAML 解析与新增步骤 shell 语法检查通过。本机 Docker Engine 仍不可连接,没有本地 Linux/Alpine 成功证据。
|
||||
|
||||
@@ -29,7 +29,7 @@ D-431 给原生 Task、Trigger、Run 列表增加页脚“下一页/回到首
|
||||
|
||||
D-433 源码候选为旧面板定时行增加“执行管理”:先展示当前 Task revision,再显式确认运行;选择确切 Run 后可以刷新、读该次日志或请求取消。它不恢复旧 Cron 写接口,也不修改定时绑定版本。网络结果不明时只重试同一 mutation,关闭窗口不会撤销已提交请求。此功能需服务端 capability 显式开放,不包含在上表任何旧归档中,见 [ADR-0535](../adr/ADR-0535-canonical-panel-run-control.md)。
|
||||
|
||||
执行管理的后续验收已接入实际面板客户端与 Linux Local API 组合门,要求读取真实进程日志、取消响应丢失后的精确重试以及重启后持久状态/日志;当前仍待 Linux 实跑。该门使用隔离数据库夹具,不是浏览器与 Owner 初始化全链路证明。阶段包可用与“原版 2.x 所有页面可直接替换后端”是不同结论,请按上表的 source revision 选择。
|
||||
执行管理的后续验收已接入实际面板客户端与 Linux Local API 组合门,要求读取真实进程日志、取消响应丢失后的精确重试以及重启后持久状态/日志。`ed4ba9b` 双架构实跑发现运行中稀疏日志不能及时读到,启动器修复及镜像回归尚待新 Linux CI 验证,不能将该候选视为可交付。该门使用隔离数据库夹具,不是浏览器与 Owner 初始化全链路证明。阶段包可用与“原版 2.x 所有页面可直接替换后端”是不同结论,请按上表的 source revision 选择。
|
||||
|
||||
D-426b2c 又补齐了 Console 镜像的 adopted-target 入口证据:切换演练使用 `ql3-local-api --cutover-probe --config <local-api.json>`,同时绑定外层 API 配置、内层 Application 配置与 exact mounts,但该模式不会启动本页使用的 listener、credential、scheduler 或 mutation surface。正常启动仍使用下文不带 `--cutover-probe` 的命令;提交 `229c3cb4e826866a0c7c4d81cb5e52cdc3975eec` 的 [artifact run 33463415938](https://github.com/whyour/qinglong/actions/runs/33463415938) 已交付 exact amd64/arm64 Console Trial Kit 与 milestone,三份下载产物的离线 auditor 均为 `compatible=true`。
|
||||
|
||||
|
||||
@@ -63,6 +63,19 @@ if [ -n "$output_quota_fifo" ]; then
|
||||
[ -z "$output_truncation_temporary" ]; then
|
||||
exit 125
|
||||
fi
|
||||
# Use each supported Linux userspace's byte-exact, streaming copy primitive.
|
||||
# GNU dd counts short reads as blocks even with count_bytes; do not use it.
|
||||
# BusyBox head can read ahead past the quota; do not use it either.
|
||||
if command -v busybox >/dev/null 2>&1 &&
|
||||
busybox dd bs=16384 count=0 iflag=count_bytes status=none </dev/null >/dev/null 2>&1; then
|
||||
output_copy=busybox
|
||||
elif head --version >/dev/null 2>&1 &&
|
||||
stdbuf -o0 head -c 0 </dev/null >/dev/null 2>&1; then
|
||||
output_copy=gnu
|
||||
else
|
||||
# No buffered or per-byte fallback on unsupported systems.
|
||||
exit 125
|
||||
fi
|
||||
rm -f "$output_truncation_temporary" 2>/dev/null || exit 125
|
||||
if ! mkfifo -m 600 "$output_quota_fifo" 2>/dev/null; then
|
||||
exit 125
|
||||
@@ -83,15 +96,25 @@ if [ -n "$output_quota_fifo" ]; then
|
||||
}
|
||||
|
||||
(
|
||||
capture_succeeded=true
|
||||
if [ "$output_quota_remaining_bytes" -gt 0 ]; then
|
||||
head -c "$output_quota_remaining_bytes"
|
||||
# Diagnostics must not bypass that quota through inherited stderr.
|
||||
if [ "$output_copy" = busybox ]; then
|
||||
busybox dd bs=16384 count="$output_quota_remaining_bytes" iflag=count_bytes status=none 2>/dev/null || capture_succeeded=false
|
||||
else
|
||||
stdbuf -o0 head -c "$output_quota_remaining_bytes" 2>/dev/null || capture_succeeded=false
|
||||
fi
|
||||
fi
|
||||
overflow_bytes=$(wc -c | tr -d '[:space:]') || overflow_bytes=
|
||||
case "$overflow_bytes" in
|
||||
''|*[!0-9]*) ;;
|
||||
0) publish_output_truncation false ;;
|
||||
*) publish_output_truncation true ;;
|
||||
esac
|
||||
# A failed copy cannot attest complete capture. Still drain the producer;
|
||||
# leave truncation unknown and retain its own exit/receipt semantics.
|
||||
if [ "$capture_succeeded" = true ]; then
|
||||
case "$overflow_bytes" in
|
||||
''|*[!0-9]*) ;;
|
||||
0) publish_output_truncation false ;;
|
||||
*) publish_output_truncation true ;;
|
||||
esac
|
||||
fi
|
||||
) < "$output_quota_fifo" &
|
||||
drain_pid=$!
|
||||
fi
|
||||
|
||||
@@ -12,7 +12,7 @@ import {
|
||||
} from './localProcessIdentity';
|
||||
|
||||
export const BUNDLED_LOCAL_PROCESS_LAUNCHER_SHA256 =
|
||||
'db4342ea57f8f7f19e385e204889ac03e97a2f82f42b01f2e59291be4b569153';
|
||||
'96653ca2b788f9a85fc3313fdf1b33b3482c121fd92b6344b26510e48a1d9804';
|
||||
export const MAX_LOCAL_PROCESS_ENVIRONMENT_ENTRIES = 256;
|
||||
export const MAX_LOCAL_PROCESS_ENVIRONMENT_BYTES = 64 * 1024;
|
||||
export const MAX_LOCAL_PROCESS_ARGUMENTS = 256;
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { spawnSync } = require('node:child_process');
|
||||
|
||||
// macOS has no production /proc identity. Tests with a fake identity provider
|
||||
// still use real GNU utilities, so BSD semantics cannot mask Linux regressions.
|
||||
function quotaEnvironment(directory) {
|
||||
if (process.platform !== 'darwin') return {};
|
||||
const bin = path.join(directory, 'quota-bin');
|
||||
fs.mkdirSync(bin, { mode: 0o700 });
|
||||
for (const [name, installed] of [['head', 'ghead'], ['stdbuf', 'stdbuf']]) {
|
||||
const found = spawnSync('/bin/sh', ['-c', `command -v ${installed}`], {
|
||||
encoding: 'utf8',
|
||||
});
|
||||
assert.equal(found.status, 0, `macOS launcher tests require coreutils ${installed}`);
|
||||
fs.symlinkSync(found.stdout.trim(), path.join(bin, name));
|
||||
}
|
||||
return { PATH: `${bin}:/usr/bin:/bin` };
|
||||
}
|
||||
|
||||
module.exports = { quotaEnvironment };
|
||||
@@ -2,6 +2,7 @@ const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
const { quotaEnvironment } = require('./helpers/quotaEnvironment.cjs');
|
||||
const { test } = require('node:test');
|
||||
const { CompletionReceiptFileStore } = require('../dist');
|
||||
const { LocalProcessLaunchError, LocalProcessLauncher } = require('../dist');
|
||||
@@ -212,6 +213,7 @@ test('hard-caps durable output and publishes an immutable truncation fact', asyn
|
||||
attemptId: ATTEMPT_ID,
|
||||
callbackSequence: 1,
|
||||
callbackToken: TOKEN,
|
||||
environment: quotaEnvironment(directory),
|
||||
command: {
|
||||
kind: 'argv',
|
||||
file: process.execPath,
|
||||
@@ -261,3 +263,162 @@ test('hard-caps durable output and publishes an immutable truncation fact', asyn
|
||||
[],
|
||||
);
|
||||
});
|
||||
|
||||
for (const kind of ['argv', 'shell']) {
|
||||
test(`publishes sparse binary ${kind} output before the process exits`, async (t) => {
|
||||
const { directory, receiptRoot } = fixture(t);
|
||||
const logArtifactId = `local-${'c'.repeat(30)}`;
|
||||
const outputFilePath = path.join(directory, `${logArtifactId}.log`);
|
||||
const release = path.join(directory, 'release');
|
||||
const marker = Buffer.from([0x00, 0xff, 0x71, 0x6c, 0x33]);
|
||||
const script = path.join(directory, 'producer.cjs');
|
||||
fs.writeFileSync(script, `
|
||||
const fs = require('node:fs');
|
||||
process.stdout.write(Buffer.from([0x00, 0xff, 0x71, 0x6c, 0x33]));
|
||||
const deadline = setTimeout(() => process.exit(92), 15000);
|
||||
const timer = setInterval(() => {
|
||||
if (!fs.existsSync(process.argv[2])) return;
|
||||
clearInterval(timer);
|
||||
clearTimeout(deadline);
|
||||
process.stderr.write('tail', () => process.exit(7));
|
||||
}, 10);
|
||||
`, { mode: 0o600 });
|
||||
const launcher = new LocalProcessLauncher(
|
||||
{ register: async () => undefined },
|
||||
{ receiptRoot, identityProvider: identityProvider() },
|
||||
);
|
||||
const quote = (value) => `'${value.replaceAll("'", "'\\''")}'`;
|
||||
const handle = await launcher.start({
|
||||
runId: RUN_ID, attemptId: ATTEMPT_ID,
|
||||
callbackSequence: 1, callbackToken: TOKEN,
|
||||
environment: quotaEnvironment(directory),
|
||||
command: kind === 'argv'
|
||||
? { kind, file: process.execPath, args: [script, release] }
|
||||
: { kind, command: [process.execPath, script, release].map(quote).join(' ') },
|
||||
output: { filePath: outputFilePath, maximumBytes: 65536, logArtifactId },
|
||||
});
|
||||
try {
|
||||
const deadline = Date.now() + 5000;
|
||||
while (fs.statSync(outputFilePath).size < marker.length && Date.now() < deadline) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
}
|
||||
assert.deepEqual(fs.readFileSync(outputFilePath), marker,
|
||||
'sparse bytes must be readable while the producer is waiting for release');
|
||||
assert.equal(await new CompletionReceiptFileStore(receiptRoot).read(ATTEMPT_ID), undefined);
|
||||
fs.writeFileSync(release, '', { mode: 0o600 });
|
||||
assert.deepEqual(await handle.completion, { exitCode: 7, signal: null });
|
||||
assert.deepEqual(fs.readFileSync(outputFilePath), Buffer.concat([marker, Buffer.from('tail')]));
|
||||
const fact = JSON.parse(fs.readFileSync(path.join(directory, `.${logArtifactId}.log.truncated.json`)));
|
||||
assert.equal(fact.quotaReached, false);
|
||||
assert.equal((await waitForReceipt(new CompletionReceiptFileStore(receiptRoot))).exitCode, 7);
|
||||
} finally {
|
||||
fs.writeFileSync(release, '', { mode: 0o600 });
|
||||
await handle.completion;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
for (const [label, initialBytes, producedBytes] of [
|
||||
['empty output', 0, 0],
|
||||
['exact quota', 0, 65536],
|
||||
['one-byte overflow', 0, 65537],
|
||||
['partial final block', 65513, 31],
|
||||
['exhausted quota', 65536, 23],
|
||||
['exhausted quota without overflow', 65536, 0],
|
||||
]) {
|
||||
test(`keeps byte-exact capture and truncation for ${label}`, async (t) => {
|
||||
const { directory, receiptRoot } = fixture(t);
|
||||
const logArtifactId = `local-${'d'.repeat(30)}`;
|
||||
const filePath = path.join(directory, `${logArtifactId}.log`);
|
||||
const initial = Buffer.alloc(initialBytes, 0x5a);
|
||||
fs.writeFileSync(filePath, initial, { mode: 0o600 });
|
||||
const produced = Buffer.from(Array.from({ length: producedBytes }, (_, i) => i % 256));
|
||||
const launcher = new LocalProcessLauncher(
|
||||
{ register: async () => undefined },
|
||||
{ receiptRoot, identityProvider: identityProvider() },
|
||||
);
|
||||
const handle = await launcher.start({
|
||||
runId: RUN_ID, attemptId: ATTEMPT_ID,
|
||||
callbackSequence: 1, callbackToken: TOKEN,
|
||||
environment: quotaEnvironment(directory),
|
||||
command: { kind: 'argv', file: process.execPath, args: ['-e', `
|
||||
const data = Buffer.from(Array.from({length: ${producedBytes}}, (_, i) => i % 256));
|
||||
let offset = 0;
|
||||
function write() {
|
||||
if (offset === data.length) return process.exit(9);
|
||||
const end = Math.min(data.length, offset + 997);
|
||||
const chunk = data.subarray(offset, end);
|
||||
offset = end;
|
||||
process.stdout.write(chunk, () => setImmediate(write));
|
||||
}
|
||||
write();
|
||||
`] },
|
||||
output: { filePath, maximumBytes: 65536, logArtifactId },
|
||||
});
|
||||
assert.deepEqual(await handle.completion, { exitCode: 9, signal: null });
|
||||
assert.deepEqual(fs.readFileSync(filePath), Buffer.concat([initial, produced]).subarray(0, 65536));
|
||||
const fact = JSON.parse(fs.readFileSync(path.join(directory, `.${logArtifactId}.log.truncated.json`)));
|
||||
assert.equal(fact.quotaReached, initialBytes + producedBytes > 65536);
|
||||
assert.equal((await waitForReceipt(new CompletionReceiptFileStore(receiptRoot))).exitCode, 9);
|
||||
});
|
||||
}
|
||||
|
||||
test('rejects unsupported capture utilities before running user code', async (t) => {
|
||||
const { directory, receiptRoot } = fixture(t);
|
||||
const bin = path.join(directory, 'unsupported-bin');
|
||||
fs.mkdirSync(bin, { mode: 0o700 });
|
||||
for (const name of ['busybox', 'head', 'stdbuf']) {
|
||||
fs.writeFileSync(path.join(bin, name), '#!/bin/sh\nexit 1\n', { mode: 0o700 });
|
||||
}
|
||||
const marker = path.join(directory, 'must-not-run');
|
||||
const logArtifactId = `local-${'e'.repeat(30)}`;
|
||||
const filePath = path.join(directory, `${logArtifactId}.log`);
|
||||
const launcher = new LocalProcessLauncher(
|
||||
{ register: async () => undefined },
|
||||
{ receiptRoot, identityProvider: identityProvider() },
|
||||
);
|
||||
const handle = await launcher.start({
|
||||
runId: RUN_ID, attemptId: ATTEMPT_ID,
|
||||
callbackSequence: 1, callbackToken: TOKEN,
|
||||
environment: { PATH: `${bin}:/usr/bin:/bin` },
|
||||
command: { kind: 'argv', file: '/usr/bin/touch', args: [marker] },
|
||||
output: { filePath, maximumBytes: 65536, logArtifactId },
|
||||
});
|
||||
assert.deepEqual(await handle.completion, { exitCode: 125, signal: null });
|
||||
assert.equal(fs.existsSync(marker), false);
|
||||
assert.equal(fs.statSync(filePath).size, 0);
|
||||
assert.equal(await new CompletionReceiptFileStore(receiptRoot).read(ATTEMPT_ID), undefined);
|
||||
assert.equal(fs.readdirSync(directory).some((name) => name.endsWith('.fifo')), false);
|
||||
});
|
||||
|
||||
test('capture failure drains output without forging a truncation fact or changing user exit', async (t) => {
|
||||
const { directory, receiptRoot } = fixture(t);
|
||||
const bin = path.join(directory, 'failing-bin');
|
||||
fs.mkdirSync(bin, { mode: 0o700 });
|
||||
fs.writeFileSync(path.join(bin, 'busybox'), `#!/bin/sh
|
||||
case " $* " in
|
||||
*' count=0 '*) exit 0 ;;
|
||||
esac
|
||||
printf 'capture failure must not bypass the log quota' >&2
|
||||
exit 1
|
||||
`, { mode: 0o700 });
|
||||
const logArtifactId = `local-${'f'.repeat(30)}`;
|
||||
const filePath = path.join(directory, `${logArtifactId}.log`);
|
||||
const launcher = new LocalProcessLauncher(
|
||||
{ register: async () => undefined },
|
||||
{ receiptRoot, identityProvider: identityProvider() },
|
||||
);
|
||||
const handle = await launcher.start({
|
||||
runId: RUN_ID, attemptId: ATTEMPT_ID,
|
||||
callbackSequence: 1, callbackToken: TOKEN,
|
||||
environment: { PATH: `${bin}:/usr/bin:/bin` },
|
||||
command: { kind: 'argv', file: process.execPath, args: [
|
||||
'-e', 'process.stdout.write(Buffer.alloc(256 * 1024), () => process.exit(9));',
|
||||
] },
|
||||
output: { filePath, maximumBytes: 65536, logArtifactId },
|
||||
});
|
||||
assert.deepEqual(await handle.completion, { exitCode: 9, signal: null });
|
||||
assert.equal(fs.statSync(filePath).size, 0);
|
||||
assert.equal(fs.existsSync(path.join(directory, `.${logArtifactId}.log.truncated.json`)), false);
|
||||
assert.equal((await waitForReceipt(new CompletionReceiptFileStore(receiptRoot))).exitCode, 9);
|
||||
});
|
||||
|
||||
@@ -4,7 +4,8 @@ const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs/promises');
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
const { test } = require('node:test');
|
||||
const { test, before, after } = require('node:test');
|
||||
const { quotaEnvironment } = require('../../ql3-local-process/test/helpers/quotaEnvironment.cjs');
|
||||
const { CompletionReceiptFileStore } = require('@qinglong/local-process');
|
||||
const {
|
||||
WorkerFileLogArtifactAllocator,
|
||||
@@ -17,6 +18,16 @@ const {
|
||||
const RUN_ID = '019f70e0-0000-7000-8000-000000000101';
|
||||
const ATTEMPT_ID = '019f70e0-0000-7000-8000-000000000102';
|
||||
const TOKEN = Buffer.alloc(32, 0x5a);
|
||||
let utilityRoot;
|
||||
let utilityEnvironment = [];
|
||||
before(async () => {
|
||||
utilityRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'ql3-worker-utilities-'));
|
||||
utilityEnvironment = Object.entries(quotaEnvironment(utilityRoot))
|
||||
.map(([name, value]) => ({ name, value }));
|
||||
});
|
||||
after(async () => {
|
||||
if (utilityRoot) await fs.rm(utilityRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
async function fixture(t) {
|
||||
const root = await fs.mkdtemp(path.join(os.tmpdir(), 'ql3-worker-posix-'));
|
||||
@@ -78,7 +89,7 @@ function launch(prepared, output, overrides = {}) {
|
||||
"process.stdout.write(process.env.QL3_RECEIPT_CALLBACK_TOKEN ? 'leaked' : 'worker-output')",
|
||||
],
|
||||
},
|
||||
environment: [],
|
||||
environment: utilityEnvironment,
|
||||
logArtifactId: prepared.logArtifactId,
|
||||
output,
|
||||
completionCallback: { sequence: 1, token: Buffer.from(TOKEN) },
|
||||
|
||||
@@ -442,6 +442,10 @@ function auditWorkflow(contents, findings) {
|
||||
'--security-opt no-new-privileges',
|
||||
'--memory=128m',
|
||||
'--pids-limit=64',
|
||||
'Verify live sparse output and byte quotas in the installed runtime',
|
||||
'for capture_image in "${IMAGE}" "${CONSOLE_IMAGE}"; do',
|
||||
'--test --test-name-pattern=\'publishes sparse|keeps byte-exact|rejects unsupported\'',
|
||||
'/opt/qinglong/node_modules/@qinglong/local-process/test/launcher.test.cjs',
|
||||
'scripts/ql3-local-image-inventory.cjs',
|
||||
'scripts/ql3-local-console-image-inventory.cjs',
|
||||
'--inventory-root=/opt/qinglong/node_modules',
|
||||
|
||||
@@ -137,6 +137,22 @@ test('rejects a Local runtime without the verified process launcher asset', () =
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects removal of the installed runtime sparse output gate', () => {
|
||||
const current = fixture();
|
||||
try {
|
||||
const workflow = path.join(current.root, '.github/workflows/ql3-ci.yml');
|
||||
fs.writeFileSync(workflow, fs.readFileSync(workflow, 'utf8').replace(
|
||||
'--test --test-name-pattern=\'publishes sparse|keeps byte-exact|rejects unsupported\'',
|
||||
'--test --test-name-pattern=\'no-tests\'',
|
||||
));
|
||||
const report = auditLocalImageContract(current.root);
|
||||
assert.equal(report.compatible, false);
|
||||
assert.ok(report.findings.some(({ code }) => code === 'LOCAL_IMAGE_CI_CONTRACT_DRIFT'));
|
||||
} finally {
|
||||
current.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects a mutable runtime base image', () => {
|
||||
const current = fixture();
|
||||
try {
|
||||
|
||||
Reference in New Issue
Block a user