mirror of
https://github.com/whyour/qinglong.git
synced 2026-10-16 22:35:32 +08:00
fix(dependencies): quote and validate Python source installs
This commit is contained in:
@@ -0,0 +1,550 @@
|
||||
# Project Architecture Guide
|
||||
|
||||
This document is written for AI coding agents and maintainers who need to understand and modify this project safely. It focuses on where behavior lives, how the application starts, and which files are usually involved for common changes.
|
||||
|
||||
## Project Summary
|
||||
|
||||
Qinglong is a timed task management platform. It provides a web admin panel for managing cron jobs, scripts, environment variables, subscriptions, dependencies, logs, configuration files, and system settings.
|
||||
|
||||
The repository is organized as a full-stack TypeScript application:
|
||||
|
||||
- `src/`: frontend admin panel, built with Umi Max, React, Ant Design, and Ant Design Pro Layout.
|
||||
- `back/`: backend application, built with Express, TypeScript, typedi, Sequelize, SQLite, gRPC, and worker processes.
|
||||
- `shell/`: runtime shell scripts used to execute tasks and preload task environments.
|
||||
- `data/`: local runtime data, including scripts, logs, configs, SQLite database, uploaded files, and cloned repositories.
|
||||
- `static/`: built frontend and backend artifacts.
|
||||
- `docker/`: Docker images, compose file, and entrypoint.
|
||||
- `sample/`: sample scripts and default config templates.
|
||||
|
||||
## High-Level Runtime Flow
|
||||
|
||||
```text
|
||||
Browser
|
||||
-> src/pages/*
|
||||
-> src/utils/http.tsx
|
||||
-> /api/*
|
||||
-> back/api/*
|
||||
-> back/services/*
|
||||
-> back/data/* Sequelize models
|
||||
-> data/db/database.sqlite
|
||||
|
||||
Cron/task execution
|
||||
-> back/services/cron.ts
|
||||
-> shell/task.sh or shell/otask.sh
|
||||
-> data/scripts/*
|
||||
-> data/log/*
|
||||
|
||||
Frontend assets in production
|
||||
-> static/dist/*
|
||||
-> served by back/loaders/express.ts
|
||||
```
|
||||
|
||||
## Main Startup Path
|
||||
|
||||
Development starts from `package.json`:
|
||||
|
||||
```bash
|
||||
pnpm start
|
||||
```
|
||||
|
||||
This runs:
|
||||
|
||||
- `start:back`: `nodemon ./back/app.ts`
|
||||
- `start:front`: `max dev`
|
||||
|
||||
Backend startup begins in `back/app.ts`.
|
||||
|
||||
Important details:
|
||||
|
||||
- The backend uses Node `cluster`.
|
||||
- The primary process initializes the database first.
|
||||
- A gRPC worker starts before the HTTP worker.
|
||||
- The HTTP worker starts Express and serves API routes plus frontend static files.
|
||||
- If the gRPC worker restarts, the HTTP worker is asked to re-register cron jobs.
|
||||
|
||||
Production-style backend output is generated by:
|
||||
|
||||
```bash
|
||||
pnpm run build:back
|
||||
```
|
||||
|
||||
The compiled backend is placed under `static/build`.
|
||||
|
||||
Frontend output is generated by:
|
||||
|
||||
```bash
|
||||
pnpm run build:front
|
||||
```
|
||||
|
||||
The compiled frontend is placed under `static/dist`.
|
||||
|
||||
## Backend Architecture
|
||||
|
||||
### Entry Point
|
||||
|
||||
- `back/app.ts`
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- Creates the Express application.
|
||||
- Starts primary/worker process logic.
|
||||
- Initializes database in the primary process.
|
||||
- Starts gRPC and HTTP workers.
|
||||
- Handles graceful shutdown.
|
||||
- Re-registers cron jobs after gRPC worker recovery.
|
||||
|
||||
### Loaders
|
||||
|
||||
- `back/loaders/app.ts`
|
||||
- `back/loaders/express.ts`
|
||||
- `back/loaders/db.ts`
|
||||
- `back/loaders/depInjector.ts`
|
||||
- `back/loaders/initData.ts`
|
||||
- `back/loaders/initFile.ts`
|
||||
- `back/loaders/initTask.ts`
|
||||
- `back/loaders/server.ts`
|
||||
- `back/loaders/sock.ts`
|
||||
|
||||
Loader responsibilities:
|
||||
|
||||
- Register dependency injection bindings.
|
||||
- Sync Sequelize models.
|
||||
- Initialize files and default data.
|
||||
- Initialize scheduled tasks.
|
||||
- Configure Express middleware.
|
||||
- Register routes.
|
||||
- Attach socket/server behavior.
|
||||
|
||||
`back/loaders/express.ts` is the main HTTP middleware and routing setup. It handles:
|
||||
|
||||
- CORS.
|
||||
- Helmet.
|
||||
- body parser.
|
||||
- static frontend serving.
|
||||
- JWT validation.
|
||||
- token validation against shared auth state.
|
||||
- `/open/*` rewrite to `/api/*`.
|
||||
- route mounting through `back/api/index.ts`.
|
||||
- frontend fallback to `static/dist/index.html`.
|
||||
- API error handling.
|
||||
|
||||
### API Routes
|
||||
|
||||
- `back/api/index.ts`
|
||||
|
||||
This file registers all API modules:
|
||||
|
||||
- `user.ts`: login, initialization, authentication-related user endpoints.
|
||||
- `env.ts`: environment variable endpoints.
|
||||
- `config.ts`: config file endpoints.
|
||||
- `log.ts`: log endpoints.
|
||||
- `cron.ts`: cron/task endpoints.
|
||||
- `script.ts`: script file endpoints.
|
||||
- `open.ts`: open API/app token endpoints.
|
||||
- `dependence.ts`: dependency management endpoints.
|
||||
- `system.ts`: system information/settings endpoints.
|
||||
- `subscription.ts`: subscription endpoints.
|
||||
- `update.ts`: update/check endpoints.
|
||||
- `health.ts`: health check endpoints.
|
||||
|
||||
Route files should stay thin. They should validate input, get a service from `typedi`'s `Container`, call the service, and return `{ code, data, message }` style responses.
|
||||
|
||||
### Services
|
||||
|
||||
- `back/services/*`
|
||||
|
||||
Services contain most business logic. Common examples:
|
||||
|
||||
- `cron.ts`: create/update/delete/run cron jobs, generate crontab data, manage logs, call scheduler client.
|
||||
- `env.ts`: manage environment variables.
|
||||
- `config.ts`: read/write config files.
|
||||
- `script.ts`: manage script files.
|
||||
- `subscription.ts`: manage script subscriptions and repository pulls.
|
||||
- `dependence.ts`: install/manage runtime dependencies.
|
||||
- `system.ts`: system info and settings.
|
||||
- `notify.ts`: notification behavior.
|
||||
- `sock.ts`: socket/log stream behavior.
|
||||
- `grpc.ts`: gRPC server lifecycle.
|
||||
- `http.ts`: HTTP server lifecycle.
|
||||
|
||||
When changing backend behavior, first find the API route, then follow it into the matching service. In most cases, the service is the right place for behavioral changes.
|
||||
|
||||
### Data Models
|
||||
|
||||
- `back/data/index.ts`
|
||||
- `back/data/*.ts`
|
||||
|
||||
The backend uses Sequelize with SQLite. Database storage is configured in `back/data/index.ts`:
|
||||
|
||||
```text
|
||||
data/db/database.sqlite
|
||||
```
|
||||
|
||||
Common model files:
|
||||
|
||||
- `cron.ts`: cron job model.
|
||||
- `cronView.ts`: saved cron table views.
|
||||
- `env.ts`: environment variable model.
|
||||
- `dependence.ts`: dependency model.
|
||||
- `open.ts`: open API app/token model.
|
||||
- `subscription.ts`: subscription model.
|
||||
- `system.ts`: system settings model.
|
||||
- `notify.ts`: notification-related data.
|
||||
|
||||
Model sync and simple column migrations are currently handled in `back/loaders/db.ts`.
|
||||
|
||||
### Configuration
|
||||
|
||||
- `back/config/index.ts`
|
||||
|
||||
This is the central runtime config file. It reads `.env`, establishes `QL_DIR`, and defines important paths:
|
||||
|
||||
- `dataPath`: runtime data root.
|
||||
- `configPath`: config files.
|
||||
- `scriptPath`: user scripts.
|
||||
- `repoPath`: subscription repositories.
|
||||
- `logPath`: task logs.
|
||||
- `dbPath`: SQLite database location.
|
||||
- `uploadPath`: uploaded files.
|
||||
- `shellPath`: shell runtime scripts.
|
||||
- `preloadPath`: JS/Python/Shell preload files.
|
||||
|
||||
Before hardcoding paths, check `back/config/index.ts`.
|
||||
|
||||
### Scheduling And gRPC
|
||||
|
||||
- `back/schedule/*`
|
||||
- `back/protos/*`
|
||||
- `back/services/grpc.ts`
|
||||
|
||||
The project has two scheduling paths:
|
||||
|
||||
- Standard crontab-style tasks are persisted and written through backend cron logic.
|
||||
- Node/gRPC scheduler logic handles cases such as second-level cron expressions or additional schedules.
|
||||
|
||||
`back/services/cron.ts` decides whether a task needs the Node scheduler using schedule shape and `extra_schedules`.
|
||||
|
||||
### Shared Backend Utilities
|
||||
|
||||
- `back/shared/*`
|
||||
- `back/config/util.ts`
|
||||
- `back/config/share.ts`
|
||||
- `back/config/http.ts`
|
||||
|
||||
Use these before adding new global helpers. Existing shared code includes:
|
||||
|
||||
- auth helpers.
|
||||
- shared store.
|
||||
- log stream manager.
|
||||
- task runner helpers.
|
||||
- concurrency limits.
|
||||
- file locking utilities.
|
||||
- HTTP/proxy helpers.
|
||||
|
||||
## Frontend Architecture
|
||||
|
||||
### Umi Config
|
||||
|
||||
- `.umirc.ts`
|
||||
|
||||
Important behavior:
|
||||
|
||||
- Dev server proxies API requests to `http://127.0.0.1:5700/`.
|
||||
- Frontend build output is `static/dist`.
|
||||
- Runtime env script is loaded from `./api/env.js`.
|
||||
- `QlBaseUrl` affects frontend public path and routing base.
|
||||
|
||||
### App Initialization
|
||||
|
||||
- `src/app.ts`
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- Load Chinese and English locale JSON.
|
||||
- Determine locale from URL/cookie/localStorage.
|
||||
- Set Umi locale.
|
||||
- Apply `QlBaseUrl` as public path and router basename.
|
||||
|
||||
### Layout And Routes
|
||||
|
||||
- `src/layouts/defaultProps.tsx`
|
||||
- `src/layouts/index.tsx`
|
||||
|
||||
`defaultProps.tsx` defines the main route/menu list. If adding a new page visible in the sidebar, update this file.
|
||||
|
||||
Current major pages:
|
||||
|
||||
- `src/pages/crontab`: timed task management.
|
||||
- `src/pages/subscription`: subscription management.
|
||||
- `src/pages/env`: environment variables.
|
||||
- `src/pages/config`: config files.
|
||||
- `src/pages/script`: script management.
|
||||
- `src/pages/dependence`: dependency management.
|
||||
- `src/pages/log`: log management.
|
||||
- `src/pages/diff`: diff tool.
|
||||
- `src/pages/setting`: system settings.
|
||||
- `src/pages/login`: login.
|
||||
- `src/pages/initialization`: first-run initialization.
|
||||
- `src/pages/error`: error page.
|
||||
|
||||
### Frontend Utilities
|
||||
|
||||
- `src/utils/http.tsx`: API request helper.
|
||||
- `src/utils/websocket.ts`: socket connection behavior.
|
||||
- `src/utils/config.ts`: frontend config helpers.
|
||||
- `src/utils/const.ts`: constants.
|
||||
- `src/utils/date.ts`: date formatting helpers.
|
||||
- `src/utils/init.ts`: initialization helpers.
|
||||
- `src/utils/codemirror/*`: CodeMirror integration.
|
||||
- `src/utils/monaco/*`: Monaco integration.
|
||||
|
||||
When changing a page's API behavior, inspect both the page file and `src/utils/http.tsx`.
|
||||
|
||||
### Components And Styling
|
||||
|
||||
- `src/components/*`: reusable UI components.
|
||||
- `src/pages/**/index.less`: page-level styles.
|
||||
- `src/pages/script/index.module.less` and `src/pages/log/index.module.less`: CSS module styles.
|
||||
- `src/assets/fonts/*`: bundled fonts.
|
||||
- `src/locales/*.json`: i18n text.
|
||||
|
||||
Follow the existing Ant Design and Ant Design Pro patterns when modifying UI.
|
||||
|
||||
## Shell Runtime
|
||||
|
||||
- `shell/task.sh`: task execution path.
|
||||
- `shell/otask.sh`: alternate/manual task execution path.
|
||||
- `shell/api.sh`: shell-side API helpers.
|
||||
- `shell/env.sh`: environment setup.
|
||||
- `shell/check.sh`: runtime check helpers.
|
||||
- `shell/update.sh`: update helpers.
|
||||
- `shell/rmlog.sh`: log cleanup.
|
||||
- `shell/share.sh`: shared shell helpers.
|
||||
- `shell/preload/*`: preload files injected into JS/Python/Shell task environments.
|
||||
|
||||
The backend often coordinates task execution, but the actual user script process environment is shaped by files in `shell/`.
|
||||
|
||||
## Runtime Data Directory
|
||||
|
||||
- `data/`
|
||||
|
||||
This directory is runtime state, not just source code. Be careful when modifying or deleting files here.
|
||||
|
||||
Important subdirectories:
|
||||
|
||||
- `data/db`: SQLite database.
|
||||
- `data/config`: generated and user-edited config files.
|
||||
- `data/scripts`: user scripts.
|
||||
- `data/repo`: cloned subscription repositories.
|
||||
- `data/log`: task logs.
|
||||
- `data/upload`: uploaded files.
|
||||
- `data/syslog`: system logs.
|
||||
- `data/ssh.d`: SSH-related runtime data.
|
||||
- `data/dep_cache`: dependency cache, when present.
|
||||
|
||||
Many bugs that appear as "backend logic" may involve state stored under `data/`.
|
||||
|
||||
## Docker And Release Files
|
||||
|
||||
- `docker/Dockerfile`
|
||||
- `docker/310.Dockerfile`
|
||||
- `docker/docker-compose.yml`
|
||||
- `docker/docker-entrypoint.sh`
|
||||
- `ecosystem.config.js`
|
||||
- `version.yaml`
|
||||
|
||||
Use these when changing deployment, container startup, PM2 behavior, or release metadata.
|
||||
|
||||
## Common Modification Map
|
||||
|
||||
### Add Or Modify A Backend API
|
||||
|
||||
Typical files:
|
||||
|
||||
1. Add or update route in `back/api/<module>.ts`.
|
||||
2. Add or update service logic in `back/services/<module>.ts`.
|
||||
3. Add or update model in `back/data/<module>.ts` if persistence changes.
|
||||
4. Add validation with `celebrate`/`Joi` near the route.
|
||||
5. Update frontend caller in `src/pages/**` or `src/utils/**`.
|
||||
|
||||
### Add A New Frontend Page
|
||||
|
||||
Typical files:
|
||||
|
||||
1. Create `src/pages/<page>/index.tsx`.
|
||||
2. Add styles in `src/pages/<page>/index.less` if needed.
|
||||
3. Register route/menu in `src/layouts/defaultProps.tsx`.
|
||||
4. Add locale strings in `src/locales/zh-CN.json` and `src/locales/en-US.json`.
|
||||
5. Add API calls through the existing request helper.
|
||||
|
||||
### Change Cron/Task Behavior
|
||||
|
||||
Start with:
|
||||
|
||||
- `back/api/cron.ts`
|
||||
- `back/services/cron.ts`
|
||||
- `back/schedule/*`
|
||||
- `shell/task.sh`
|
||||
- `shell/otask.sh`
|
||||
- `shell/preload/*`
|
||||
|
||||
Also inspect:
|
||||
|
||||
- `back/data/cron.ts`
|
||||
- `back/validation/schedule.ts`
|
||||
- `data/config/crontab.list`
|
||||
- `data/log/*`
|
||||
|
||||
### Change Environment Variable Behavior
|
||||
|
||||
Start with:
|
||||
|
||||
- `back/api/env.ts`
|
||||
- `back/services/env.ts`
|
||||
- `back/data/env.ts`
|
||||
- `src/pages/env/index.tsx`
|
||||
|
||||
Also inspect:
|
||||
|
||||
- `shell/preload/env.sh`
|
||||
- `shell/preload/env.js`
|
||||
- `shell/preload/env.py`
|
||||
|
||||
### Change Script Management
|
||||
|
||||
Start with:
|
||||
|
||||
- `back/api/script.ts`
|
||||
- `back/services/script.ts`
|
||||
- `src/pages/script/index.tsx`
|
||||
- `data/scripts/*`
|
||||
|
||||
### Change Login/Auth/Security
|
||||
|
||||
Start with:
|
||||
|
||||
- `back/api/user.ts`
|
||||
- `back/services/user.ts`
|
||||
- `back/shared/auth.ts`
|
||||
- `back/shared/store.ts`
|
||||
- `back/loaders/express.ts`
|
||||
- `back/token.ts`
|
||||
- `src/pages/login/index.tsx`
|
||||
- `src/pages/initialization/index.tsx`
|
||||
|
||||
Be careful with:
|
||||
|
||||
- JWT behavior.
|
||||
- open API token behavior.
|
||||
- first-run initialization.
|
||||
- platform-specific session limits.
|
||||
|
||||
### Change Subscription Behavior
|
||||
|
||||
Start with:
|
||||
|
||||
- `back/api/subscription.ts`
|
||||
- `back/services/subscription.ts`
|
||||
- `back/data/subscription.ts`
|
||||
- `src/pages/subscription/index.tsx`
|
||||
- `data/repo/*`
|
||||
|
||||
### Change Dependency Management
|
||||
|
||||
Start with:
|
||||
|
||||
- `back/api/dependence.ts`
|
||||
- `back/services/dependence.ts`
|
||||
- `back/data/dependence.ts`
|
||||
- `src/pages/dependence/index.tsx`
|
||||
- `data/deps`
|
||||
- `data/dep_cache`
|
||||
|
||||
### Change Logs Or Live Log Streaming
|
||||
|
||||
Start with:
|
||||
|
||||
- `back/api/log.ts`
|
||||
- `back/services/log.ts`
|
||||
- `back/services/sock.ts`
|
||||
- `back/shared/logStreamManager.ts`
|
||||
- `src/pages/log/index.tsx`
|
||||
- `src/components/terminal.tsx`
|
||||
- `data/log/*`
|
||||
|
||||
## Coding Conventions
|
||||
|
||||
Backend:
|
||||
|
||||
- Prefer adding business logic to services, not route files.
|
||||
- Use `typedi` services consistently.
|
||||
- Use existing config paths from `back/config/index.ts`.
|
||||
- Return API responses in the existing `{ code, data, message }` shape.
|
||||
- Use existing utilities before adding new helpers.
|
||||
- Preserve current SQLite/Sequelize style unless doing a larger data-layer refactor.
|
||||
|
||||
Frontend:
|
||||
|
||||
- Follow existing Umi/React/Ant Design patterns.
|
||||
- Keep route/menu changes in `src/layouts/defaultProps.tsx`.
|
||||
- Use existing request/WebSocket helpers.
|
||||
- Add or update locale strings for visible UI text.
|
||||
- Keep page-specific styles near the page.
|
||||
|
||||
Shell/runtime:
|
||||
|
||||
- Treat `shell/` as part of production behavior.
|
||||
- Test task execution changes with realistic scripts when possible.
|
||||
- Be careful with path quoting and environment variable propagation.
|
||||
|
||||
Data:
|
||||
|
||||
- Treat `data/` as mutable runtime state.
|
||||
- Do not delete runtime state unless explicitly requested.
|
||||
- Schema changes should account for existing SQLite databases.
|
||||
|
||||
## Suggested First Steps For AI Agents
|
||||
|
||||
When asked to modify behavior:
|
||||
|
||||
1. Identify whether the change is frontend, backend, shell runtime, data model, or deployment.
|
||||
2. Search by feature name in `src/pages`, `back/api`, and `back/services`.
|
||||
3. Read the route file and matching service before editing.
|
||||
4. If persistence is involved, read the matching `back/data` model and `back/loaders/db.ts`.
|
||||
5. If task execution is involved, inspect `shell/` and `back/services/cron.ts`.
|
||||
6. Make the smallest scoped change that matches existing patterns.
|
||||
7. Run the most relevant check:
|
||||
- `pnpm run build:back` for backend TypeScript changes.
|
||||
- `pnpm run build:front` for frontend build changes.
|
||||
- targeted manual task/API checks for shell and scheduler changes.
|
||||
|
||||
## Quick Directory Reference
|
||||
|
||||
```text
|
||||
.
|
||||
├── back/ Backend TypeScript application
|
||||
│ ├── api/ Express route modules
|
||||
│ ├── config/ Runtime config, paths, constants, helpers
|
||||
│ ├── data/ Sequelize models and SQLite connection
|
||||
│ ├── loaders/ Startup initialization and Express setup
|
||||
│ ├── middlewares/ Express middlewares
|
||||
│ ├── protos/ gRPC proto files and generated TS
|
||||
│ ├── schedule/ Scheduler/gRPC client helpers
|
||||
│ ├── services/ Business logic services
|
||||
│ ├── shared/ Shared backend utilities
|
||||
│ └── validation/ Joi validation schemas
|
||||
├── src/ Frontend Umi/React application
|
||||
│ ├── assets/ Fonts and static frontend assets
|
||||
│ ├── components/ Shared UI components
|
||||
│ ├── hooks/ Frontend hooks
|
||||
│ ├── layouts/ Main layout and menu route config
|
||||
│ ├── locales/ i18n JSON
|
||||
│ ├── pages/ Feature pages
|
||||
│ └── utils/ HTTP, WebSocket, editor, date, and config utilities
|
||||
├── shell/ Task runtime shell scripts and preload files
|
||||
├── data/ Runtime state: db, logs, scripts, repos, configs
|
||||
├── docker/ Docker build and compose files
|
||||
├── sample/ Sample scripts and default config templates
|
||||
├── static/ Built frontend/backend artifacts
|
||||
└── docs/ Project documentation
|
||||
```
|
||||
@@ -0,0 +1,49 @@
|
||||
# Node cron 调度器迁移
|
||||
|
||||
Node 调度路径使用固定版本 `node-cron@4.6.0`,普通任务(含额外规则)、订阅和系统 cron 共用 `back/shared/cronScheduler.ts`。启动/一次性任务、间隔任务、脚本并发限制和进程执行链路保持原有实现。系统模式只将语义一致的普通五字段规则交给 crond。
|
||||
|
||||
移除 `node-schedule`、其类型包以及兼容别名 `cron-parser-v4`。保留 `cron-parser@5` 做旧表达式的校验与标准化;计时器和日历推进由 node-cron 管理。
|
||||
|
||||
## 兼容行为
|
||||
|
||||
- 保留旧版 1–6 字段补全、五种宏、月份/星期名称、数字起点步长、L 和 #。另外明确新增 `@annually`(等同 `@yearly`)、`@midnight`(等同 `@daily`)、`@minutely`(每分钟第 0 秒)。宏按进程时区解释。`?` 仅允许日期和星期字段;裸 `/N`、H 和其他未列出的宏仍拒绝。
|
||||
- 日期与星期同时指定时保留 OR 语义,不采用 node-cron 默认的 AND。两个原生任务分别处理日期和星期,并按计划时间过滤交集,避免重复执行;保留旧解析器的月份全日期范围及全局 nth-week 限制。
|
||||
- 原生任务先暂停创建并验证,再替换旧任务。构造失败时销毁已准备的实例并保留旧快照。
|
||||
- 进程内迟到的触发通过 `execution:missed` 加入既有执行队列,记录补执行日志。并发和单任务积压上限仍由青龙队列控制;这不提供进程退出期间的持久化补跑或分布式 exactly-once 保证。
|
||||
- 标准化结果使用上限 512 条的只读缓存,仅缓存语法,不缓存运行实例或下次执行时间。避免大批相同表达式重复解析导致恢复 RPC 超时。
|
||||
- 回调异常记录日志,后续调度继续;取消时调用原生 `destroy()` 释放计时器和注册表。
|
||||
- 持久化无效任务在恢复时跳过并记录日志,无效订阅不会产生未处理的启动拒绝。API、导入和执行使用同一套表达式校验。
|
||||
|
||||
## 复审修复
|
||||
|
||||
- 移除没有生产调用的 `nextInvocation()` 接口,避免对 OR/# 等过滤规则报告不会实际执行的原生日期。回归测试直接观察实际执行和原生实例状态。
|
||||
- 批量注册 RPC 时限为 5 秒加每条规则 5 毫秒,计入额外规则并封顶 120 秒;保留超时后的恢复机制,不立即重放结果不确定的写入。
|
||||
- Alpine/系统模式下,数字起点步长(如 `0/5`)、星期包含 `7`、日期步长、日期和星期同时受限的表达式转入 Node 路径。BusyBox 的这些解析或通配语义与旧 Node 解析器不同,不能仅因是纯数字五字段就交给 crond。
|
||||
- 这些规则在 `crontab.list` 中仍保留为注释,避免同时被系统和 Node 执行。其他可移植规则仍使用系统 crond。
|
||||
|
||||
## 夏令时差异
|
||||
|
||||
采用 node-cron 的当地时间语义:春季跳时不存在的时间跳过,秋季重复小时不再执行第二次。旧 cron-parser 对部分春季时间会顺延,并可能重复秋季高频任务。使用有夏令时的时区时需要考虑该行为变化;Asia/Shanghai 不受影响。回归测试固定 America/New_York 的 2026 年转换日期。
|
||||
|
||||
## 验证
|
||||
|
||||
`npm test` 和 `npm run build:back`。
|
||||
|
||||
- `test/fixtures/legacy-cron.json` 是迁移前使用 node-schedule 2.1.1 / cron-parser 4.9.0 保存的快照:269 条语法判定,18 组日历序列,每组 10 次执行。
|
||||
- 真正的 node-cron 实例配合虚拟时钟验证日期/星期 OR、交集去重、闰年、L/#、8 秒迟到补执行、取消释放、回调拒绝和夏令时。
|
||||
- 注册构造失败回滚、无效订阅隔离、空表达式立即运行,以及现有调度恢复/503/子进程测试覆盖接入流程。
|
||||
- 隔离的正式 Debian 镜像使用实际编译产物验证 10,000 条有效任务和 2 条错误任务的恢复、300 个同时到期的 Python 脚本、并发上限 16 及 HTTP 健康检查。具体结果见本次测试报告;这是生产镜像隔离验证,不是线上长期运行证明。
|
||||
|
||||
### 本次验收结果(2026-09-27)
|
||||
|
||||
完整回归 260 通过、3 项原有跳过,后端编译成功。Node 20 生产运行时的 21 项兼容测试通过。隔离镜像中 300/300 脚本完成,无重复或遗漏,最大并发 16,270 个成功及 30 个预设失败均落库;完成耗时 52.36 秒。恢复后健康检查 1,870/1,870 成功,任务列表 38/38 成功。重启恢复阶段另有 50 次健康检查失败。
|
||||
|
||||
初版重复解析曾导致恢复超过 5 秒 RPC 时限,加入有上限的语法缓存后重跑通过。本次 10,000 个任务共享两个表达式,不能视为一万种复杂表达式的容量保证;实际脚本耗时也不代表纯调度性能。
|
||||
|
||||
新增宏在 UTC 和 Asia/Shanghai 下验证连续两次实际触发及不提前触发;原有语法快照仅对这三个新增宏允许预期变化。
|
||||
|
||||
### 复审后验收
|
||||
|
||||
完整回归 267 通过、3 项原有跳过;Node 20 的 10 项修复回归及 Alpine 镜像的 24 项兼容/宏测试通过。
|
||||
|
||||
额外使用 9,700 个不同表达式和 300 个同时到期任务验证恢复(另有 2 个无效任务),不再局限于共享两个表达式。300/300 脚本完成,无重复或遗漏,最大并发 16,270 个退出码 0、30 个预设退出码 3 均落库。恢复后健康检查 1,650/1,650 成功、任务列表 33/33 成功;启动恢复阶段另有 260 次健康检查失败,整批脚本耗时 60.27 秒。较大快照恢复仍需等待,不承诺启动阶段始终可用或任意规模都能在时限内恢复。
|
||||
@@ -0,0 +1,7 @@
|
||||
# Python 依赖来源
|
||||
|
||||
依赖管理中可手动添加 Python 包名、`git+https://` 等 pip 支持的源 URL,或脚本/订阅仓库目录内的 requirements 文件、pyproject.toml 路径。文件路径支持目录空格,必须位于持久化 scripts 或 repo 根目录内;越界与指向根目录外的符号链接会被拒绝。
|
||||
|
||||
requirements 使用 `pip install -r`,pyproject 使用完整项目目录安装,保留相对依赖文件和项目源码。拉取订阅不会自动安装仓库声明的依赖。
|
||||
|
||||
删除 requirements 条目会调用 `pip uninstall -r`。源 URL 或 pyproject 条目无法可靠逆向推断其安装的所有分发包,因此删除该条目只移除面板记录;需另外按包名卸载已安装的包,日志会明确提示。
|
||||
@@ -0,0 +1,78 @@
|
||||
# Task startup and notification loading
|
||||
|
||||
`task` performs shell configuration, status reporting and dependency-path discovery
|
||||
before invoking the selected runtime. These costs are separate from cron callback
|
||||
latency and time spent waiting for an execution slot.
|
||||
|
||||
## Dependency-path cache
|
||||
|
||||
Node scheduling usually starts in `/ql`, while Alpine crond starts in `/root`.
|
||||
The pnpm discovery key includes the working directory, npm/pnpm configuration,
|
||||
relevant environment and executable metadata. Each key now has its own cached
|
||||
record, so alternating scheduler environments do not evict each other.
|
||||
|
||||
Records expire after 60 seconds. Refreshes share a persistent per-user lock;
|
||||
records older than an hour are removed on refresh. Warm lookups do not scan the
|
||||
cache directory. The lock is never removed while another process might use it.
|
||||
The opt-out `QL_NODE_PATH_CACHE=0`, failed-discovery fallback, and package resolution
|
||||
order remain available. This caches the dependency directory, not its contents.
|
||||
|
||||
## Language startup paths
|
||||
|
||||
| Invocation | Environment and before hooks | Automatic notification loading |
|
||||
| --- | --- | --- |
|
||||
| `task script.py` / `.pyc` | Python preload imports generated environment, runs the shell/command before hooks, imports `task_before.py`, then applies account selection | `QLAPI.notify(...)` imports `__ql_notify__` on first call |
|
||||
| `task script.js` / `.mjs` / `.ts` | Node preload imports generated environment, runs shell/command before hooks, requires `task_before.js`, then applies account selection | `QLAPI.notify(...)` requires `__ql_notify__.js` on first call |
|
||||
| `task script.sh` | Shell sources the generated environment and runs shell before hooks | No automatic Python/Node notification module |
|
||||
| Explicit interpreter or other command, e.g. `task python3 script.py` | Retains generated environment and shell before-hook exports; it does not automatically install the Python/Node preload | No automatic notification module unless configured by the caller |
|
||||
|
||||
TypeScript uses `ts-node-transpile-only`; MJS uses the same Node preload plus the
|
||||
ESM loader. There are no separate built-in Ruby, Go, Java or PHP preload modules.
|
||||
All these task paths still pass through shared dependency-path setup: non-JS
|
||||
scripts and their hooks may themselves call Node or use the QLAPI bridge.
|
||||
|
||||
The before hooks are deliberately still synchronous: they can export environment
|
||||
variables or perform work required by the user's script. Python's preload still
|
||||
starts a child shell/Python to capture that environment; Node's preload starts a
|
||||
child shell/Node. Node also still loads its gRPC client eagerly. These are remaining
|
||||
startup costs, not work performed by the cron library, and are not removed by this
|
||||
change.
|
||||
|
||||
Shell scripts run in a child shell that inherits the wrapper's environment and
|
||||
before-hook functions. This lets the wrapper run its after hooks and report the
|
||||
exit status even when a script calls `exit`, replaces itself with `exec`, or fails
|
||||
under `set -e`. The script's own EXIT trap still runs in the child. Changes made
|
||||
inside the script (variables, working directory, functions and shell options)
|
||||
do not propagate back to the wrapper or its after hooks.
|
||||
|
||||
## Lazy notifications
|
||||
|
||||
Scripts keep calling `QLAPI.notify` with the same arguments. When no notification
|
||||
is sent, the notification provider and its channel dependencies are never imported
|
||||
by the preload. First use loads the provider; subsequent uses reuse the standard
|
||||
Python/Node module cache. JS forwards the receiver and return value, including
|
||||
promises, and Python forwards positional and keyword arguments. Import/send errors
|
||||
surface at the notification call rather than preventing unrelated scripts from
|
||||
obtaining QLAPI during startup. A failed import can be retried after its cause is
|
||||
resolved.
|
||||
|
||||
Provider initialization, including provider configuration reads, now happens at
|
||||
first use. A script's explicit `import __ql_notify__`, `require(...)`, or independent
|
||||
notification helper still loads that module immediately.
|
||||
|
||||
## Task timeouts
|
||||
|
||||
`task -m DURATION ...` also applies to sourced Shell scripts. The timeout worker
|
||||
inherits before-hook variables, arrays and functions without serializing them.
|
||||
At the deadline, the worker's process group receives TERM, followed by KILL after
|
||||
0.5 seconds, so ordinary child processes cannot continue after the timeout.
|
||||
Explicit interpreters and other commands use the same mechanism. Processes that
|
||||
intentionally create a separate session/process group are outside this boundary.
|
||||
The wrapper records status 124 and runs its after hook; concurrent account mode
|
||||
retains its existing aggregate status behavior. Zero disables the timeout.
|
||||
|
||||
Cron executions waiting for a concurrency slot retain their schedule revision.
|
||||
Disabling, deleting or replacing that revision prevents its queued executions
|
||||
from starting when a slot becomes available. Already running executions retain
|
||||
the existing stop behavior. Invalidated callbacks do not count against a new
|
||||
revision's limit of five pending/running executions.
|
||||
@@ -0,0 +1,49 @@
|
||||
# 2.22.0 发布准备验收
|
||||
|
||||
日期:2026-09-26。候选代码基于 `origin/develop@801a71d7402f5d4b582318ed88ad00633357a531`,主包和面板版本统一为 `2.22.0`。本文件记录验收证据,不代表已经发布。
|
||||
|
||||
## 自动化与构建
|
||||
|
||||
- 新增 5 项日志弹窗回归:大日志结束后读完分页、尾部截取不重复取历史、运行中空日志继续轮询、关闭取消定时器/丢弃在途响应、旧响应和不前进偏移不死循环。
|
||||
- 用真实日志读取器写入 600 KiB 后结束任务:原 develop 在新增回归中失败,修复后 5 项全通过。
|
||||
- macOS arm64、Node 20、按冻结锁文件独立安装依赖:228 项测试,225 通过、3 项平台相关跳过、0 失败。后端 TypeScript 与前端生产构建通过;前端有已有的大包体提示。
|
||||
- `npm pack --dry-run` 成功,包含前后端产物,主包版本为 2.22.0;npm registry 查询该版本返回 E404(尚未占用,发布时仍需重新核实)。
|
||||
- 上游基线 Linux CI 为 223 项回归全部通过;CLI Node 22.12/24 流水线通过,Node 24 的 291 项测试全部通过。该结果属于上游提交,不冒充本分支新的 CI 结果。
|
||||
- GitNexus 对 `getCronLog` 修改前影响分析:LOW,直接调用者 `CronLogModal`,1 条流程。最终差异分析涉及日志弹窗流程;行号移动也将 `autoScroll` 列出,但该函数实现未修改。
|
||||
|
||||
## 容器升级与回退
|
||||
|
||||
测试仅使用新建账号、新建任务和隔离容器数据,不接触真实用户数据。
|
||||
|
||||
- 已发布旧版镜像 `whyour/qinglong:2.21.0` digest:`sha256:e72f77d855a86cfabf774de76b030731c38540b251a0c888038810fbcf256c48`。镜像内源码为 `9b37f064d815baf49b1869445bbd60e4209bacc9`,与当前 master 提交不同;这是已发布镜像到候选版的实际升级验收。
|
||||
- 候选运行环境为 `whyour/qinglong:develop` digest:`sha256:35c5b9f2d0666ab7fcde954f175087c097e93fa852306360aa07b8aa0a5d0481`,revision 标签对应 `801a71d7402f5d4b582318ed88ad00633357a531`。装入本分支本地构建的前后端及版本文件。因此验证了候选代码的容器运行,不等同于最终发行镜像的完整构建验收。
|
||||
- 旧版创建账号及真实 Shell 脚本任务并验证独立输出,停机复制完整 `/ql/data` 作为一致的升级前备份。
|
||||
- 候选版读取该备份后原密码可登录、原任务仍存在并能真实执行,数据库已增加迁移表,测试账号密码已转为 scrypt(仅检查布尔值,未记录凭据)。
|
||||
- 长运行脚本启动后确认实际 PID;停止后确认该进程不存在;立即重跑并再次停止成功。状态字段恢复为空闲;不把历史 PID 字段当作进程存活判断。
|
||||
- 两步验证启用、密码阶段返回挑战、OTP 登录、同一验证码重放拒绝、关闭两步验证及重新登录均通过。
|
||||
- 每 5 秒调度的测试脚本产生持续输出;本容器 HTTP 夹具订阅下载 V1 文件成功。
|
||||
- 订阅夹具更新为 V2 后再次同步,落盘内容已变为 V2。
|
||||
- 容器重启前调度标记 34 行,重启后 41 行,HTTP/gRPC 健康且可重新登录,确认定时注册恢复。
|
||||
- 将升级前完整备份恢复到全新的 2.21.0 容器,原密码登录、原任务保留及真实脚本输出均通过。
|
||||
|
||||
## 生产依赖审计:仍需处理
|
||||
|
||||
在无共享 node_modules 的临时目录,使用候选 `package.json` 和 `pnpm-lock.yaml` 执行 `pnpm audit --prod --json`:414 个依赖,报告 3 Critical、38 High、49 Moderate、9 Low。数量是扫描公告命中,不等于独立漏洞数量或均能在面板中利用。审计时间与数据库变化会影响结果。
|
||||
|
||||
| Critical 命中 | 生产依赖路径 | 初步适用性与处理 |
|
||||
| --- | --- | --- |
|
||||
| protobufjs 7.5.4 | `@grpc/proto-loader → protobufjs` | 公告要求攻击者能控制加载的 schema/descriptor;当前主要使用仓库内生成的协议,尚未证实外部可控 schema 入口。仍应升级到修复版本并验证 gRPC。 |
|
||||
| websocket-driver 0.7.4 | `sockjs → websocket-driver` / `faye-websocket` | 处理网络协议帧,不能用面板业务鉴权替代协议层修复;应升级到至少 0.7.5 并回归 WebSocket。 |
|
||||
| tar 6.2.1 | `@whyour/sqlite3 → tar`、`@mapbox/node-pre-gyp → tar` | 主要涉及安装/构建解压链,不能直接等同于面板上传接口漏洞。修复跨 tar 主版本,需验证 SQLite 预编译下载和源码编译路径;另有 tar 7.5.20 的后续公告。 |
|
||||
|
||||
参考:[protobufjs 公告](https://github.com/advisories/GHSA-xq3m-2v4x-88gg)、[websocket-driver 公告](https://github.com/advisories/GHSA-xv26-6w52-cph6)、[tar 公告](https://github.com/advisories/GHSA-23hp-3jrh-7fpw)。
|
||||
|
||||
High 命中还涉及 gRPC、multer、js-yaml、JWT 的 jws、undici 等运行时依赖。当前准备改动没有批量升级依赖;不能宣称安全扫描通过。正式发布仍需完成依赖升级与可达性评估、回归,或由维护者明确记录接受的风险。
|
||||
|
||||
## 尚未放行的发布步骤
|
||||
|
||||
- 本分支新的 Linux CI 尚待执行。
|
||||
- Python 3.10 镜像和全部架构的最终候选构建尚未验证;上游 develop 工作流跳过这些发布专用任务。
|
||||
- 主包及 CLI 的 npm Trusted Publisher 设置属于 npm 账户配置,未通过代码或只读公开 registry 验证;不得把 `id-token: write` 视作已完成授权。
|
||||
- 最终发版时确认 `publishTime` 与实际发布时间一致,并重新核对版本是否可发布。
|
||||
- 完成上述事项前不合并到 master、不打正式 tag、不执行 `shell/pub.sh`。
|
||||
@@ -0,0 +1,62 @@
|
||||
# 青龙 2.22.0
|
||||
|
||||
## 2026-09-27 覆盖更新
|
||||
|
||||
- 修复历史任务中合法的 `?` 定时表达式被拒绝,导致启动恢复失败、面板健康检查返回 503 的问题(#3078)。主规则和附加规则均保留兼容性,无效规则仍在替换现有任务前拒绝。
|
||||
- 修复 `ql check` 等 Shell 诊断输出以 `--` 开头时的 `printf: --: invalid option`,补全调度注册失败日志中的错误详情。
|
||||
- 启动恢复跳过无效主规则或附加规则的任务,记录任务 ID、名称和错误规则;数据库原记录保留可编辑,其他任务与面板正常启动。服务连接故障仍返回 503。
|
||||
- 校验使用与实际调度器一致的解析器,避免接受 `H` 等无法执行的语法;五字段扩展规则、宏和历史简写交给 Node 调度器,避免系统 crontab 拒绝整个任务快照。
|
||||
- 已对 250 多个表达式样本与真实调度库做一致性校验,验证无效任务隔离和两种调度模式;本地回归 240 项,237 通过、3 项平台相关跳过,后端构建通过。
|
||||
|
||||
## 新增功能
|
||||
|
||||
- 独立远程命令行包 `@whyour/qinglong-cli`:覆盖任务、订阅、环境变量、配置、脚本、日志、依赖、应用、系统等 OpenAPI,支持认证、JSON 输出和中英文帮助。
|
||||
- 可选面板内部 TypeScript 工具:任务执行、订阅同步、启动检查、升级重载和账号恢复。默认继续使用原 Shell 入口。
|
||||
- 仪表盘支持点击今日成功、今日失败查看任务明细。
|
||||
- 客户端 IP 和代理链诊断、可信代理配置、登录 IP 黑名单。
|
||||
- 历史运行实例和统计数据的保留参数、清理预览、手动清理、依赖缓存清理和数据库压缩。保存保留参数不等于开启自动清理。
|
||||
- WPUSH 通知,支持面板、JavaScript、Python 和 gRPC。
|
||||
- `ql log [--lines <n>] [--nostream]` 查看服务日志。
|
||||
|
||||
## 修复与优化
|
||||
|
||||
- 修复任务生命周期、调度锁、调度器就绪和恢复、排队与停止竞争、订阅清理,以及停止失败的反馈。
|
||||
- 日志改为分块读取;任务结束后继续读完剩余分页,确保最后的输出和退出信息可见。修复中文任务日志名。
|
||||
- 加固密码存储、JWT 有效性校验、两步验证、认证状态变更和文件路径访问。
|
||||
- 修复过期登录令牌引起的页面刷新循环,完善任务字段校验提示。
|
||||
- 减少常驻依赖和重复 WebSocket 鉴权,限制指标缓存和日志显示量,优化大目录枚举。
|
||||
- 优化 Node 路径缓存与 Alpine 并发缓存锁,修正依赖缓存清理后的安装状态。
|
||||
- 启动时保留 `.env` 配置;保留容器启动错误诊断;数据库迁移失败时中止启动。
|
||||
- 镜像增加 provenance、SBOM、源码/静态产物/依赖锁文件一致性校验,改进多架构构建。
|
||||
|
||||
## 升级兼容性
|
||||
|
||||
### 数据备份与回退
|
||||
|
||||
升级前停止面板和任务,备份完整持久化数据目录,并记录当前镜像的不可变 digest、启动参数、挂载和环境配置。不要只复制正在写入的 SQLite 主文件;必须保留完整离线数据备份。
|
||||
|
||||
新版兼容旧密码;首次成功登录或修改密码后,密码会迁移为 scrypt 哈希。旧版不识别这种格式,因此**不能只回退镜像并继续使用升级后的数据**。
|
||||
|
||||
需要回退时:停止新版,另外保存升级后的数据,恢复升级前的完整备份,再使用原镜像 digest 和原启动配置启动旧版。恢复备份会丢弃备份后产生的任务、配置和日志变更。如必须保留升级后数据,应单独验证旧版兼容性,并通过可信终端使用旧版账号恢复功能重新设置密码;不要手工把哈希当作密码使用。
|
||||
|
||||
数据库迁移新增 `SchemaMigrations` 和任务排队标识;迁移异常会阻止启动。应检查启动日志并修复数据或恢复备份,不要通过删除数据库来绕过错误。
|
||||
|
||||
### API 与日志
|
||||
|
||||
旧文件读取路由 `/configs/:file`、`/scripts/:file`、`/logs/:file` 返回业务码 `410`,请改用对应的 `/detail` 接口(适用于 `/api` 和 `/open` 前缀)。
|
||||
|
||||
日志接口默认返回尾部片段。客户端需要使用 `offset`、`nextOffset`、`total` 读取后续数据;任务已结束不代表所有日志已经读取完毕。面板仅保留有限长度的日志显示,完整文件仍在日志目录中。
|
||||
|
||||
修改账号或两步验证配置会使旧会话失效;客户端应重新登录。IP 黑名单针对登录;反向代理场景应先在客户端 IP 页面确认识别结果再配置黑名单。
|
||||
|
||||
### CLI
|
||||
|
||||
远程 npm CLI 要求 Node >=22.12,推荐 Node 24。它与面板内置工具均使用 `ql` 名称;已有面板的机器应使用独立 npm prefix 或 `npm exec --package=@whyour/qinglong-cli -- ql ...`,避免覆盖内置命令。
|
||||
|
||||
远程 npm 包不包含本机执行器、升级、重载和账号恢复实现,也不会自动切换面板入口。使用内部工具需单独构建完整 `cli/dist`,在满足 Node 版本要求的环境中显式设置 `QL_CLI_ROOT`;清除该变量并重启恢复 Shell 入口。标准面板镜像并未默认启用内部 TypeScript 工具。
|
||||
|
||||
## 发布验收
|
||||
|
||||
正式发布前应完成:旧版离线数据升级、登录与两步验证、定时/手动任务、停止后立即重跑、订阅更新、容器重启调度恢复和备份回退。发布者还需检查 Python 3.10 镜像矩阵,以及主包和 CLI 的 npm Trusted Publisher 配置。
|
||||
|
||||
依赖审计与执行证据见同目录的 `2.22.0-validation.md`。通过单元测试或构建不代表完成所有平台和真实升级场景的验收。
|
||||
@@ -0,0 +1,33 @@
|
||||
# 青龙 2.23.0
|
||||
|
||||
## 新增功能
|
||||
|
||||
- 脚本历史版本:查看、对比和恢复历史内容;每个脚本保留最多 20 个版本,支持不超过 1 MiB 的 UTF-8 文本脚本。恢复前校验当前内容哈希,发生并发修改时返回冲突,避免覆盖新内容。
|
||||
- 独立远程 CLI 0.1.2 增加脚本历史查看、对比和恢复命令,并完善资源冲突及认证错误反馈。最低要求仍为 Node.js 22.12。
|
||||
|
||||
## 修复与优化
|
||||
|
||||
- 普通任务、额外规则和订阅的 Node 调度路径迁移到 node-cron。保留旧规则标准化、日期/星期 OR、L/# 等兼容行为,补充 `@annually`、`@midnight`、`@minutely`。
|
||||
- 禁用、删除或修改任务后,旧规则对应的排队执行不会再启动。完善任务进程超时、停止回收、Shell before/after 环境传递和退出状态。
|
||||
- 修复系统令牌未主动续期和认证缓存并发更新:定时生成与启动后重新计时会主动续期,避免长期运行后失效。Shell API 每次请求检查令牌,遇到 401 时刷新并重试一次;其他错误不自动重试。
|
||||
- 降低任务启动开销;JS/Python 的 QLAPI 通知依赖在首次通知调用时加载,减少不发送通知的任务启动成本。
|
||||
- 加固环境变量名和字面值、订阅参数、路径访问及 SSH 配置输入;保留合法的中文及分支别名兼容。
|
||||
- 修复页面升级、`ql update` 和静态资源的版本同步,完善升级暂存校验及应用、完整备份与恢复流程。
|
||||
- 修复 Debian 上的依赖安装、通知和调度回归,补齐前端缺失的中英文翻译,改进 Gitee/GitLab 静态资源镜像同步检查。
|
||||
- 最小升级 compression、multer、@grpc/grpc-js、proxy-addr、websocket-driver、sequelize,修补现有 HTTP、上传、代理信任、WebSocket、gRPC 和 JSON 查询路径相关风险。完整依赖治理、Node/pnpm 迁移及工具打包调整延期。
|
||||
|
||||
## 升级说明
|
||||
|
||||
升级前停止面板与任务,备份完整持久化数据目录,包括数据库、脚本、配置、日志、依赖缓存及已有的 `script-history` 目录,并记录原镜像摘要和启动配置。使用面板备份接口时,确认选择了脚本、配置和日志;仅备份数据库不能恢复全部数据。
|
||||
|
||||
升级后试运行代表性的 JavaScript、Python、Shell 任务,检查订阅、环境变量、依赖安装以及通知。存在夏令时的时区采用 node-cron 行为:跳过春季不存在的当地时间,秋季重复小时不重复执行;Asia/Shanghai 不受影响。调度器不提供进程退出期间的持久化补跑保证。
|
||||
|
||||
任务超时会终止普通子进程所在的进程组并记录退出状态 124;主动创建独立会话/进程组的程序不在该边界内。并发账号执行模式保留既有汇总退出状态。
|
||||
|
||||
本次沿用现有 Dockerfile 和 pnpm 8.3.1。默认 Debian 镜像使用 Debian 12 与 Node.js 20,默认 Alpine 镜像的 Node.js 由 APK 提供;默认 Python 为 3.11。Node 20 与 Node 18 构建运行时已结束上游维护,相关迁移延期,功能验证通过不代表全部依赖风险清零。
|
||||
|
||||
2.23.0 是 Python 3.10 兼容镜像的最后一个版本。`2.23.0-python3.10` 和 `2.23.0-debian-python3.10` 发布后停止维护,不再提供后续更新;请验证脚本和依赖后迁移至默认 Python 3.11 镜像。
|
||||
|
||||
主包 `@whyour/qinglong` 与独立远程包 `@whyour/qinglong-cli` 的 `ql` 命令用途不同。在已有面板的主机上使用独立 npm prefix 或 `npm exec --package=@whyour/qinglong-cli -- ql ...`,避免覆盖面板内置入口。
|
||||
|
||||
发布时间:2026-10-11 12:27:02(Asia/Shanghai)。本版本已完成四类镜像验证及独立 dev 环境超过两小时的持续功能验收。
|
||||
@@ -0,0 +1,53 @@
|
||||
# 脚本编辑历史
|
||||
|
||||
脚本管理工具栏的“历史版本”打开右侧抽屉。选择版本后,对比历史内容与服务器上当前已保存文件;桌面使用双栏对比,手机使用行内对比。恢复需要确认,有未保存修改时必须先保存或取消。恢复前保留当前版本,因此恢复操作也可以撤销。普通保存不再弹出不可恢复确认。
|
||||
|
||||
## 记录范围
|
||||
|
||||
- 编辑器 `PUT /api/scripts` 和调试窗口 `POST /api/scripts` 覆盖已有脚本时,在内容变化时留档,首次修改前的内容也保留;相同内容不写历史。
|
||||
- 历史只包含保存前真实存在的内容;不把失败写入当成一个内容版本。
|
||||
- 上传、新增、重命名、删除、订阅更新和外部编辑不主动留档。若下次面板保存时检测到外部变更,会保留那一刻的内容,标记为“面板外修改后”。这不代表外部修改之前的内容受到保护。
|
||||
- 按脚本规范化后的相对路径隔离历史;不同目录同名文件不会混淆,指向同一脚本的安全符号链接共用历史。重命名后不会自动迁移历史。调试器另存为覆盖已有脚本时,目标旧内容进入历史,删除源文件前将其旧内容保留到现有 bak 目录;备份失败则不覆盖目标、不删除源文件。源和目标实际指向同一脚本时(如安全符号链接或不同路径写法),按原地保存处理,保留两个路径,不删除源文件。历史不能用于回收已删除的文件。
|
||||
- 历史记录支持至多 1 MiB 的 UTF-8 文本文件(含空文件)。当前文件或新内容超出范围时,界面要求明确确认“不记录历史直接保存”;取消则保持文件和草稿不变。确认后以 `skipHistory: true` 和服务端返回的当前哈希重试,锁内检查哈希以防确认期间覆盖他人修改。成功后明确提示本次未留档。大文件可以通过此流程缩小,后续恢复普通留档。未确认的 API 请求返回 413,并包含 `historyUnavailable` 和 `currentHash`;历史读取损坏等其他错误不允许用此选项绕过。
|
||||
|
||||
## 存储与资源边界
|
||||
|
||||
位置:`<QL_DATA_DIR>/script-history/<SHA-256(规范相对路径)>.br`。目录位于脚本目录之外,避免进入脚本树、订阅和任务扫描。目录权限 `0700`,历史文件权限 `0600`。目录随整个数据目录备份。
|
||||
|
||||
每个脚本仅一个 Brotli 压缩 JSON 包,存储完整的旧内容快照以及时间、来源、内容哈希。新文件内容不再重复存储,只保存头版本元数据,供下一次留档标注来源。面板服务通过 TypeDI 注册为单例,路由使用 `Container.get(ScriptHistoryService)` 获取;构造实例只初始化配置,首次发生保存时才创建历史目录。包内重复文本共享压缩窗口,不依赖补丁链,也不需要额外数据库、文件监听、后台任务或常驻缓存。使用 Node 内置 zlib 异步压缩,Brotli quality=4、window=4 MiB,不新增依赖。
|
||||
|
||||
| 限制 | 默认值 |
|
||||
| --- | --- |
|
||||
| 每脚本历史版本 | 最多 20 个 |
|
||||
| 单个脚本内容 | 1 MiB |
|
||||
| 单包解压后的 JSON | 8 MiB |
|
||||
| 单个压缩历史包 | 2 MiB |
|
||||
| 所有压缩历史包合计 | 32 MiB |
|
||||
| 有历史的脚本数 | 最多 1000 个 |
|
||||
| 历史写入暂存文件 | 共享一个,最多 2 MiB |
|
||||
| 脚本写入暂存文件 | 每次保存一个,与本次保存内容等大,结束后清理 |
|
||||
|
||||
达到单文件限制时删除最旧版本,至少保留本次覆盖前的内容;无法保留则拒绝覆盖。达到全局限制时,在脚本成功提交后优先清理最久未更新的脚本历史包(该文件的整份历史),不删除脚本本身。历史暂存、提交或脚本提交失败时不清理任何其他历史;脚本提交失败会回滚本文件历史。如果提交成功但清理失败,返回 `cleanupPending`,界面提示检查存储权限和空间,后续保存重新尝试清理。配额是正常清理成功后的目标,事务期间和清理失败时会暂时超出。32 MiB 是文件内容字节数,不包含文件系统块分配、目录和锁的开销。清理只在发生保存时执行,通常最多扫描 1000 个包的元数据(清理失败期间可能更多),不解压其他脚本的历史。读取仅解压当前脚本,限制解压输出长度。
|
||||
|
||||
文件保存复用现有锁路径,同时以独立跨进程目录锁串行化历史写入和配额清理;`finally` 释放锁。先准备并同步脚本暂存文件,保留文件权限和所有者;随后写并同步历史暂存文件、原子提交历史,再原子替换脚本。任一步准备失败不会截断原脚本;最后替换脚本失败时还原此前历史;成功后才清理配额。超限确认保存不创建新历史包,使用同样的脚本暂存和替换方式,并流式计算旧文件哈希。外部进程不遵守该锁时,不承诺与其并发写入的事务一致性。
|
||||
|
||||
恢复请求携带预览时的当前内容哈希,锁内检查后才写入;文件变化则返回 409。历史读写与脚本读取采用相同的路径和符号链接边界检查。面板配置目录中的敏感文件仍受黑名单保护;脚本目录中的 auth.json、token.json 等普通脚本名称可以正常保存和留档。
|
||||
|
||||
## 压缩样本
|
||||
|
||||
运行 `node scripts/benchmark-script-history.cjs` 可复现比较。模拟同一脚本连续 20 次小修改,比较独立 gzip、合并 gzip 与合并 Brotli。首次本机测量:
|
||||
|
||||
| 样本 | 独立 gzip | 合并 gzip | 合并 Brotli(quality 4) |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| sample/notify.js | 242,430 B | 221,832 B | 13,930 B |
|
||||
| shell/task.sh | 30,159 B | 2,317 B | 1,737 B |
|
||||
|
||||
这是文本重复度较高的模拟样本,不是所有工作负载的压缩保证。Brotli 合包特别适合大小超过 gzip 32 KiB 窗口、但每次只改少量内容的脚本。增量补丁可能在部分样本更小,但增加补丁生成、链式恢复、裁剪和损坏恢复的复杂度;本实现选择压缩合包与硬上限,在磁盘、CPU 和可靠性之间取平衡。
|
||||
|
||||
## API
|
||||
|
||||
- `GET /api/scripts/history?filename=...&path=...`:版本元数据、当前哈希、保留数量。
|
||||
- `GET /api/scripts/history/detail?filename=...&path=...&id=...`:历史内容、当前内容及当前哈希。
|
||||
- `PUT /api/scripts/history/restore`:`{filename, path, id, expectedHash}`。
|
||||
|
||||
沿用现有 scripts 路由的认证与权限。历史 ID 使用 UUID;不存在或已清理返回 404,文件已变更返回 409,文本或容量限制返回 413。
|
||||
Reference in New Issue
Block a user