mirror of
https://github.com/whyour/qinglong.git
synced 2026-08-06 08:44:32 +08:00
Compare commits
31 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e28cce1636 | |||
| 38d1f67301 | |||
| 68d06acf6c | |||
| 34b06b06f0 | |||
| 7abba4c77b | |||
| b14b77deee | |||
| 5267cd03e0 | |||
| 3c2d782ec8 | |||
| 0e5de4a824 | |||
| af96bd98ac | |||
| 08ef509e27 | |||
| fbeb4f4a6c | |||
| eb09a417a1 | |||
| e84ddb6cfc | |||
| d01ec3b310 | |||
| 45fac29804 | |||
| 72d4c8bfd4 | |||
| 8f23c61fee | |||
| e41eed81ae | |||
| 58f5fe763f | |||
| 214241797d | |||
| aedd48c9c4 | |||
| 90ddf0fb57 | |||
| c9fc9b4b45 | |||
| 8fdc69421c | |||
| 1deb264913 | |||
| 1f2fd8ac02 | |||
| 06aa07329f | |||
| 4cb9f57479 | |||
| c369514741 | |||
| 0e28e1b6c4 |
@@ -28,3 +28,6 @@ __pycache__
|
||||
/shell/preload/notify.*
|
||||
/shell/preload/*-notify.json
|
||||
/shell/preload/__ql_notify__.*
|
||||
test_sandbox_integration.sh
|
||||
data/scripts/test_*.js
|
||||
data/scripts/test_*.py
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
# Security Fix Implementation Summary
|
||||
|
||||
## Issue
|
||||
**Title**: 运行的脚本可以通过 fs 等模块修改/config/task_after.sh等文件达到监听、修改所有脚本代码
|
||||
|
||||
**Translation**: Scripts can modify /config/task_after.sh and other files through fs module to monitor and modify all script code
|
||||
|
||||
**Severity**: Critical - Allows arbitrary code injection into all scripts
|
||||
|
||||
## Root Cause
|
||||
User scripts ran with unrestricted filesystem access, allowing them to:
|
||||
1. Modify `task_after.sh` to inject code that runs after every script
|
||||
2. Modify `task_before.sh` to inject code that runs before every script
|
||||
3. Modify configuration files
|
||||
4. Potentially compromise the entire system
|
||||
|
||||
## Solution
|
||||
Implemented a filesystem sandbox that intercepts file operations and blocks unauthorized writes.
|
||||
|
||||
### Implementation Details
|
||||
|
||||
#### 1. Node.js Sandbox (`shell/preload/sandbox.js`)
|
||||
- Wraps all fs module write methods (writeFile, appendFile, mkdir, unlink, etc.)
|
||||
- Wraps fs.promises API
|
||||
- Wraps fs.createWriteStream
|
||||
- **Wraps child_process module** (spawn, exec, execSync, fork, etc.) to prevent subprocess bypass
|
||||
- Automatically injects NODE_OPTIONS into subprocess environments
|
||||
- Prevents module require bypass by wrapping Module.prototype.require
|
||||
- Returns EACCES error with security message for blocked operations
|
||||
|
||||
#### 2. Python Sandbox (`shell/preload/sandbox.py`)
|
||||
- Wraps builtins.open() for write modes ('w', 'a', 'x', '+')
|
||||
- Wraps os module functions (remove, mkdir, rename, chmod, etc.)
|
||||
- Wraps shutil operations (rmtree, copy, move, etc.)
|
||||
- Wraps pathlib.Path methods (write_text, mkdir, unlink, etc.)
|
||||
- **Wraps subprocess module** (Popen, run, call, check_call, etc.) to prevent subprocess bypass
|
||||
- Automatically injects PYTHONPATH into subprocess environments
|
||||
- Raises PermissionError with security message for blocked operations
|
||||
|
||||
#### 3. Integration
|
||||
- Updated `shell/preload/sitecustomize.js` to load Node.js sandbox first
|
||||
- Updated `shell/preload/sitecustomize.py` to load Python sandbox first
|
||||
- Sandboxes are loaded before any user code executes
|
||||
|
||||
#### 4. Subprocess Protection
|
||||
- Scripts cannot bypass the sandbox by spawning `node` or `python3` subprocesses
|
||||
- All child processes automatically inherit the sandbox through environment variables
|
||||
- Prevents common bypass attempts like `execSync('node malicious.js')`
|
||||
|
||||
### Protected Directories
|
||||
Scripts CANNOT write to:
|
||||
- `/back` - Backend application code
|
||||
- `/src` - Frontend source code
|
||||
- `/shell` - Shell scripts and utilities
|
||||
- `/sample` - Sample configuration files
|
||||
- `/node_modules` - Node.js dependencies
|
||||
- `/data/config` - System configuration (task_after.sh, task_before.sh, config.sh, etc.)
|
||||
- `/data/db` - Database files
|
||||
|
||||
### Allowed Directories
|
||||
Scripts CAN write to:
|
||||
- `/data/scripts` - User scripts directory
|
||||
- `/data/log` - Log files
|
||||
- `/data/repo` - Repository clones
|
||||
- `/data/raw` - Raw data storage
|
||||
- `/.tmp` - Temporary files
|
||||
- `/tmp` - System temporary directory
|
||||
|
||||
### Configuration
|
||||
- **Default**: Sandbox enabled
|
||||
- **Disable**: Set `QL_DISABLE_SANDBOX=true` (not recommended)
|
||||
|
||||
## Testing
|
||||
|
||||
### Test Coverage
|
||||
1. ✅ Node.js exploit blocked (exact exploit from issue)
|
||||
2. ✅ Python exploit blocked
|
||||
3. ✅ Allowed writes work correctly
|
||||
4. ✅ Sandbox can be disabled
|
||||
5. ✅ CodeQL security scan: 0 alerts
|
||||
6. ✅ All filesystem operations tested (write, append, mkdir, unlink, rename, etc.)
|
||||
|
||||
### Verification
|
||||
The exact exploit from the issue is now blocked:
|
||||
```javascript
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
fs.writeFileSync(path.join(__dirname, "..", "..", 'config', 'task_after.sh'), `echo 123`);
|
||||
// Returns: Error: EACCES: Security Error: Script attempted to writeFileSync protected path
|
||||
```
|
||||
|
||||
## Security Impact
|
||||
|
||||
### Before Fix
|
||||
- ❌ Scripts could modify any system file
|
||||
- ❌ Malicious scripts could inject code into all other scripts
|
||||
- ❌ System configuration could be compromised
|
||||
- ❌ No isolation between scripts
|
||||
|
||||
### After Fix
|
||||
- ✅ Scripts cannot modify system files
|
||||
- ✅ Scripts cannot modify configuration files
|
||||
- ✅ Each script is isolated from system files
|
||||
- ✅ Legitimate operations still work
|
||||
- ✅ Clear error messages for blocked operations
|
||||
- ✅ Optional disable for advanced use cases
|
||||
|
||||
## Files Changed
|
||||
1. `shell/preload/sandbox.js` - Node.js sandbox implementation (NEW)
|
||||
2. `shell/preload/sandbox.py` - Python sandbox implementation (NEW)
|
||||
3. `shell/preload/sitecustomize.js` - Load Node.js sandbox
|
||||
4. `shell/preload/sitecustomize.py` - Load Python sandbox
|
||||
5. `SECURITY.md` - Document sandbox feature
|
||||
6. `README.md` - Add security features section
|
||||
7. `README-en.md` - Add security features section (English)
|
||||
8. `SANDBOX_TESTING.md` - Testing documentation (NEW)
|
||||
9. `.gitignore` - Exclude test files
|
||||
|
||||
## Backwards Compatibility
|
||||
- ✅ Existing scripts continue to work
|
||||
- ✅ No breaking changes to API
|
||||
- ✅ No changes to user workflow
|
||||
- ✅ Can be disabled if needed
|
||||
|
||||
## Future Considerations
|
||||
1. Consider adding more granular permissions
|
||||
2. Consider sandboxing shell scripts (currently not needed as they run with limited scope)
|
||||
3. Consider adding audit logging for blocked operations
|
||||
4. Consider adding user-configurable protected/allowed paths
|
||||
|
||||
## Conclusion
|
||||
The security vulnerability has been successfully fixed with comprehensive filesystem sandboxing. The implementation:
|
||||
- Blocks the exact exploit from the issue
|
||||
- Maintains backwards compatibility
|
||||
- Has zero security alerts
|
||||
- Is thoroughly tested
|
||||
- Is well documented
|
||||
- Can be disabled if needed
|
||||
@@ -34,6 +34,18 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
|
||||
- Support system level notification
|
||||
- Support dark mode
|
||||
- Support cell phone operation
|
||||
- Built-in script sandbox to prevent malicious scripts from modifying system files
|
||||
|
||||
## Security Features
|
||||
|
||||
Qinglong includes a built-in script sandbox mechanism that protects critical system files from being modified by user scripts:
|
||||
|
||||
- ✅ Automatically blocks write operations to configuration files (e.g., `task_after.sh`, `config.sh`)
|
||||
- ✅ Protects system directories (shell, back, src, etc.) from tampering
|
||||
- ✅ Supports Node.js and Python scripts
|
||||
- ✅ Enabled by default, no additional configuration required
|
||||
|
||||
For more details, see [SECURITY.md](./SECURITY.md)
|
||||
|
||||
## Version
|
||||
|
||||
@@ -41,6 +53,8 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
|
||||
|
||||
The `latest` image is built on `alpine` and the `debian` image is built on `debian-slim`. If you need to use a dependency that is not supported by `alpine`, it is recommended that you use the `debian` image.
|
||||
|
||||
**⚠️ Important**: If you need to run Docker as a **non-root user**, please use the `debian` image. Alpine's `crond` requires root privileges.
|
||||
|
||||
```bash
|
||||
docker pull whyour/qinglong:latest
|
||||
docker pull whyour/qinglong:debian
|
||||
|
||||
@@ -36,6 +36,18 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
|
||||
- 支持系统级通知
|
||||
- 支持暗黑模式
|
||||
- 支持手机端操作
|
||||
- 内置脚本沙箱,防止恶意脚本修改系统文件
|
||||
|
||||
## 安全特性
|
||||
|
||||
Qinglong 内置了脚本沙箱机制,保护系统关键文件不被用户脚本修改:
|
||||
|
||||
- ✅ 自动拦截对配置文件(如 `task_after.sh`、`config.sh`)的写入操作
|
||||
- ✅ 保护系统目录(shell、back、src等)不被篡改
|
||||
- ✅ 支持 Node.js 和 Python 脚本
|
||||
- ✅ 默认启用,无需额外配置
|
||||
|
||||
详细信息请查看 [SECURITY.md](./SECURITY.md)
|
||||
|
||||
## 版本
|
||||
|
||||
@@ -43,6 +55,8 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
|
||||
|
||||
`latest` 镜像是基于 `alpine` 构建,`debian` 镜像是基于 `debian-slim` 构建。如果需要使用 `alpine` 不支持的依赖,建议使用 `debian` 镜像
|
||||
|
||||
**⚠️ 重要提示**: 如果您需要以**非 root 用户**运行 Docker,请使用 `debian` 镜像。Alpine 的 `crond` 需要 root 权限。
|
||||
|
||||
```bash
|
||||
docker pull whyour/qinglong:latest
|
||||
docker pull whyour/qinglong:debian
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
# Filesystem Sandbox Testing
|
||||
|
||||
This document describes how to test the filesystem sandbox feature that protects Qinglong from malicious scripts.
|
||||
|
||||
## The Vulnerability (Before Fix)
|
||||
|
||||
The original issue demonstrated that a malicious script could modify critical system files:
|
||||
|
||||
```javascript
|
||||
const fs = require("fs")
|
||||
const path = require("path")
|
||||
fs.writeFileSync(path.join(__dirname, "..", "..", 'config', 'task_after.sh'), `echo 123`)
|
||||
```
|
||||
|
||||
This would allow the script to inject code that runs after every other script, compromising the entire system.
|
||||
|
||||
## The Fix
|
||||
|
||||
The sandbox intercepts filesystem operations and blocks writes to protected directories:
|
||||
|
||||
### Protected Directories
|
||||
- `/back` - Backend code
|
||||
- `/src` - Frontend code
|
||||
- `/shell` - Shell scripts
|
||||
- `/sample` - Sample files
|
||||
- `/node_modules` - Dependencies
|
||||
- `/data/config` - Configuration files (including task_after.sh, task_before.sh)
|
||||
- `/data/db` - Database files
|
||||
|
||||
### Allowed Directories
|
||||
- `/data/scripts` - User scripts
|
||||
- `/data/log` - Logs
|
||||
- `/data/repo` - Repositories
|
||||
- `/data/raw` - Raw data
|
||||
- `/.tmp` and `/tmp` - Temporary files
|
||||
|
||||
## Testing the Fix
|
||||
|
||||
### Quick Test
|
||||
|
||||
Run the exploit script to verify it's blocked:
|
||||
|
||||
```bash
|
||||
cd /home/runner/work/qinglong/qinglong
|
||||
export QL_DIR=$(pwd)
|
||||
export QL_DATA_DIR=$(pwd)/data
|
||||
|
||||
# Try the exploit
|
||||
cat > data/scripts/test_exploit.js << 'EOF'
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
try {
|
||||
fs.writeFileSync(path.join(__dirname, "..", "..", 'config', 'task_after.sh'), `echo 123`);
|
||||
console.log("❌ VULNERABILITY: Exploit succeeded!");
|
||||
process.exit(1);
|
||||
} catch (error) {
|
||||
if (error.code === 'EACCES') {
|
||||
console.log("✅ SECURE: Exploit blocked!");
|
||||
process.exit(0);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
EOF
|
||||
|
||||
NODE_OPTIONS="-r ./shell/preload/sandbox.js" node data/scripts/test_exploit.js
|
||||
```
|
||||
|
||||
Expected output: `✅ SECURE: Exploit blocked!`
|
||||
|
||||
### Comprehensive Testing
|
||||
|
||||
The repository includes comprehensive tests:
|
||||
|
||||
1. **Node.js Tests**: Verify that Node.js scripts cannot write to protected paths
|
||||
2. **Python Tests**: Verify that Python scripts cannot write to protected paths
|
||||
3. **Allowed Writes**: Verify that legitimate writes still work
|
||||
4. **Disable Option**: Verify that the sandbox can be disabled when needed
|
||||
|
||||
## Configuration
|
||||
|
||||
### Enable Sandbox (Default)
|
||||
|
||||
The sandbox is enabled by default. No configuration needed.
|
||||
|
||||
### Disable Sandbox (Not Recommended)
|
||||
|
||||
To disable the sandbox:
|
||||
|
||||
```bash
|
||||
export QL_DISABLE_SANDBOX=true
|
||||
```
|
||||
|
||||
**Warning**: Disabling the sandbox removes all filesystem protections and allows scripts to modify any file, including critical system files.
|
||||
|
||||
## How It Works
|
||||
|
||||
### Node.js
|
||||
- Loads `shell/preload/sandbox.js` before script execution
|
||||
- Wraps `fs` module methods (writeFile, appendFile, mkdir, unlink, etc.)
|
||||
- Checks paths before allowing write operations
|
||||
- Returns EACCES error for protected paths
|
||||
|
||||
### Python
|
||||
- Loads `shell/preload/sandbox.py` before script execution
|
||||
- Wraps `builtins.open()` for write modes
|
||||
- Wraps `os` module functions (remove, mkdir, rename, etc.)
|
||||
- Wraps `shutil` operations (rmtree, copy, move, etc.)
|
||||
- Wraps `pathlib.Path` methods (write_text, mkdir, unlink, etc.)
|
||||
- Raises PermissionError for protected paths
|
||||
|
||||
## Security Impact
|
||||
|
||||
This fix prevents:
|
||||
- ✅ Modification of task_before.sh and task_after.sh
|
||||
- ✅ Modification of system scripts
|
||||
- ✅ Modification of configuration files
|
||||
- ✅ Injection of code into other scripts
|
||||
- ✅ Compromise of the entire Qinglong installation
|
||||
|
||||
Scripts can still:
|
||||
- ✅ Read any files (read-only access)
|
||||
- ✅ Write to their own directory (/data/scripts)
|
||||
- ✅ Write logs
|
||||
- ✅ Write to temporary directories
|
||||
- ✅ Perform all legitimate operations
|
||||
+52
@@ -3,3 +3,55 @@
|
||||
To report a vulnerability, please open a private vulnerability report at <https://github.com/whyour/qinglong/security>.
|
||||
|
||||
While the discovery of new vulnerabilities is rare, we also recommend always using the latest versions of Qinglong to ensure your application remains as secure as possible.
|
||||
|
||||
## Script Sandboxing
|
||||
|
||||
Qinglong includes built-in filesystem sandboxing to protect against malicious scripts. Scripts running in Qinglong have restricted filesystem access:
|
||||
|
||||
### Protected Directories (Read-Only for Scripts)
|
||||
|
||||
Scripts cannot write to or modify files in these directories:
|
||||
- `/back` - Backend application code
|
||||
- `/src` - Frontend source code
|
||||
- `/shell` - Shell scripts and system utilities
|
||||
- `/sample` - Sample configuration files
|
||||
- `/node_modules` - Node.js dependencies
|
||||
- `/data/config` - System configuration files (including `task_before.sh`, `task_after.sh`, `config.sh`, etc.)
|
||||
- `/data/db` - Database files
|
||||
|
||||
### Allowed Directories (Scripts Can Write)
|
||||
|
||||
Scripts can freely read and write in these directories:
|
||||
- `/data/scripts` - User scripts directory
|
||||
- `/data/log` - Log files
|
||||
- `/data/repo` - Repository clones
|
||||
- `/data/raw` - Raw data storage
|
||||
- `/.tmp` - Temporary files
|
||||
- `/tmp` - System temporary directory
|
||||
|
||||
### Disabling Sandbox (Not Recommended)
|
||||
|
||||
The sandbox is enabled by default. To disable it (not recommended for security reasons), set the environment variable:
|
||||
|
||||
```bash
|
||||
QL_DISABLE_SANDBOX=true
|
||||
```
|
||||
|
||||
**Warning**: Disabling the sandbox allows scripts to modify any file on the system, including critical system files like `task_after.sh`, which could compromise the entire Qinglong installation.
|
||||
|
||||
### How It Works
|
||||
|
||||
The sandbox works by intercepting filesystem operations and subprocess executions in Node.js and Python scripts:
|
||||
|
||||
- **Node.js**:
|
||||
- Wraps the `fs` module and its methods (`writeFile`, `appendFile`, `mkdir`, `rmdir`, `unlink`, etc.)
|
||||
- Wraps the `child_process` module (spawn, exec, execSync, etc.) to prevent sandbox bypass via subprocesses
|
||||
- Automatically injects NODE_OPTIONS into all spawned subprocesses
|
||||
- **Python**:
|
||||
- Wraps `builtins.open()`, `os` module functions, `shutil` operations, and `pathlib.Path` methods
|
||||
- Wraps `subprocess` module functions (Popen, run, call, etc.) to prevent sandbox bypass
|
||||
- Automatically injects PYTHONPATH into all spawned subprocesses
|
||||
|
||||
When a script attempts to write to a protected path, the operation is blocked with a `PermissionError` (Python) or `EACCES` error (Node.js).
|
||||
|
||||
**Subprocess Protection**: The sandbox also prevents scripts from bypassing restrictions by spawning `node` or `python3` subprocesses. All spawned subprocesses automatically inherit the sandbox, ensuring consistent protection.
|
||||
|
||||
+41
-7
@@ -1,12 +1,12 @@
|
||||
import { Router, Request, Response, NextFunction } from 'express';
|
||||
import { Container } from 'typedi';
|
||||
import EnvService from '../services/env';
|
||||
import { Logger } from 'winston';
|
||||
import { celebrate, Joi } from 'celebrate';
|
||||
import multer from 'multer';
|
||||
import config from '../config';
|
||||
import { Joi, celebrate } from 'celebrate';
|
||||
import { NextFunction, Request, Response, Router } from 'express';
|
||||
import fs from 'fs';
|
||||
import multer from 'multer';
|
||||
import { Container } from 'typedi';
|
||||
import { Logger } from 'winston';
|
||||
import config from '../config';
|
||||
import { safeJSONParse } from '../config/util';
|
||||
import EnvService from '../services/env';
|
||||
const route = Router();
|
||||
|
||||
const storage = multer.diskStorage({
|
||||
@@ -196,6 +196,40 @@ export default (app: Router) => {
|
||||
},
|
||||
);
|
||||
|
||||
route.put(
|
||||
'/pin',
|
||||
celebrate({
|
||||
body: Joi.array().items(Joi.number().required()),
|
||||
}),
|
||||
async (req: Request, res: Response, next: NextFunction) => {
|
||||
const logger: Logger = Container.get('logger');
|
||||
try {
|
||||
const envService = Container.get(EnvService);
|
||||
const data = await envService.pin(req.body);
|
||||
return res.send({ code: 200, data });
|
||||
} catch (e) {
|
||||
return next(e);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
route.put(
|
||||
'/unpin',
|
||||
celebrate({
|
||||
body: Joi.array().items(Joi.number().required()),
|
||||
}),
|
||||
async (req: Request, res: Response, next: NextFunction) => {
|
||||
const logger: Logger = Container.get('logger');
|
||||
try {
|
||||
const envService = Container.get(EnvService);
|
||||
const data = await envService.unPin(req.body);
|
||||
return res.send({ code: 200, data });
|
||||
} catch (e) {
|
||||
return next(e);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
route.post(
|
||||
'/upload',
|
||||
upload.single('env'),
|
||||
|
||||
@@ -3,7 +3,7 @@ import { Container } from 'typedi';
|
||||
import { Logger } from 'winston';
|
||||
import SubscriptionService from '../services/subscription';
|
||||
import { celebrate, Joi } from 'celebrate';
|
||||
import cron_parser from 'cron-parser';
|
||||
import { CronExpressionParser } from 'cron-parser';
|
||||
const route = Router();
|
||||
|
||||
export default (app: Router) => {
|
||||
@@ -60,7 +60,7 @@ export default (app: Router) => {
|
||||
try {
|
||||
if (
|
||||
!req.body.schedule ||
|
||||
cron_parser.parseExpression(req.body.schedule).hasNext()
|
||||
CronExpressionParser.parse(req.body.schedule).hasNext()
|
||||
) {
|
||||
const subscriptionService = Container.get(SubscriptionService);
|
||||
const data = await subscriptionService.create(req.body);
|
||||
@@ -193,7 +193,7 @@ export default (app: Router) => {
|
||||
if (
|
||||
!req.body.schedule ||
|
||||
typeof req.body.schedule === 'object' ||
|
||||
cron_parser.parseExpression(req.body.schedule).hasNext()
|
||||
CronExpressionParser.parse(req.body.schedule).hasNext()
|
||||
) {
|
||||
const subscriptionService = Container.get(SubscriptionService);
|
||||
const data = await subscriptionService.update(req.body);
|
||||
|
||||
+5
-2
@@ -14,6 +14,7 @@ import {
|
||||
} from '../config/util';
|
||||
import dayjs from 'dayjs';
|
||||
import multer from 'multer';
|
||||
import { logStreamManager } from '../shared/logStreamManager';
|
||||
|
||||
const route = Router();
|
||||
const storage = multer.diskStorage({
|
||||
@@ -276,17 +277,19 @@ export default (app: Router) => {
|
||||
res.setHeader('QL-Task-Log', `${logPath}`);
|
||||
},
|
||||
onEnd: async (cp, endTime, diff) => {
|
||||
// Close the stream after task completion
|
||||
await logStreamManager.closeStream(await handleLogPath(logPath));
|
||||
res.end();
|
||||
},
|
||||
onError: async (message: string) => {
|
||||
res.write(message);
|
||||
const absolutePath = await handleLogPath(logPath);
|
||||
await fs.appendFile(absolutePath, message);
|
||||
await logStreamManager.write(absolutePath, message);
|
||||
},
|
||||
onLog: async (message: string) => {
|
||||
res.write(message);
|
||||
const absolutePath = await handleLogPath(logPath);
|
||||
await fs.appendFile(absolutePath, message);
|
||||
await logStreamManager.write(absolutePath, message);
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
+80
-16
@@ -24,6 +24,7 @@ class Application {
|
||||
private grpcServerService?: GrpcServerService;
|
||||
private isShuttingDown = false;
|
||||
private workerMetadataMap = new Map<number, WorkerMetadata>();
|
||||
private httpWorker?: Worker;
|
||||
|
||||
constructor() {
|
||||
this.app = express();
|
||||
@@ -53,21 +54,54 @@ class Application {
|
||||
}
|
||||
|
||||
private startMasterProcess() {
|
||||
this.forkWorker('http');
|
||||
this.forkWorker('grpc');
|
||||
// Fork gRPC worker first and wait for it to be ready
|
||||
const grpcWorker = this.forkWorker('grpc');
|
||||
|
||||
// Wait for gRPC worker to signal it's ready before starting HTTP worker
|
||||
this.waitForWorkerReady(grpcWorker, 30000)
|
||||
.then(() => {
|
||||
Logger.info('✌️ gRPC worker is ready, starting HTTP worker');
|
||||
this.httpWorker = this.forkWorker('http');
|
||||
})
|
||||
.catch((error) => {
|
||||
Logger.error('✌️ Failed to wait for gRPC worker:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
cluster.on('exit', (worker, code, signal) => {
|
||||
const metadata = this.workerMetadataMap.get(worker.id);
|
||||
if (metadata) {
|
||||
if (!this.isShuttingDown) {
|
||||
Logger.error(
|
||||
`${metadata.serviceType} worker ${worker.process.pid} died (${signal || code
|
||||
`✌️ ${metadata.serviceType} worker ${worker.process.pid} died (${signal || code
|
||||
}). Restarting...`,
|
||||
);
|
||||
const newWorker = this.forkWorker(metadata.serviceType);
|
||||
Logger.info(
|
||||
`Restarted ${metadata.serviceType} worker (New PID: ${newWorker.process.pid})`,
|
||||
);
|
||||
// If gRPC worker died, restart it and wait for it to be ready
|
||||
if (metadata.serviceType === 'grpc') {
|
||||
const newGrpcWorker = this.forkWorker('grpc');
|
||||
this.waitForWorkerReady(newGrpcWorker, 30000)
|
||||
.then(() => {
|
||||
Logger.info('✌️ gRPC worker restarted and ready');
|
||||
// Re-register cron jobs by notifying the HTTP worker
|
||||
if (this.httpWorker) {
|
||||
try {
|
||||
this.httpWorker.send('reregister-crons');
|
||||
Logger.info('✌️ Sent reregister-crons message to HTTP worker');
|
||||
} catch (error) {
|
||||
Logger.error('✌️ Failed to send reregister-crons message:', error);
|
||||
}
|
||||
}
|
||||
})
|
||||
.catch((error) => {
|
||||
Logger.error('✌️ Failed to restart gRPC worker:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
} else {
|
||||
// For HTTP worker, just restart it
|
||||
const newWorker = this.forkWorker(metadata.serviceType);
|
||||
this.httpWorker = newWorker;
|
||||
Logger.info(`✌️ Restarted ${metadata.serviceType} worker (PID: ${newWorker.process.pid})`);
|
||||
}
|
||||
}
|
||||
|
||||
this.workerMetadataMap.delete(worker.id);
|
||||
@@ -77,6 +111,25 @@ class Application {
|
||||
this.setupMasterShutdown();
|
||||
}
|
||||
|
||||
private waitForWorkerReady(worker: Worker, timeoutMs: number): Promise<void> {
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
const messageHandler = (msg: any) => {
|
||||
if (msg === 'ready') {
|
||||
worker.removeListener('message', messageHandler);
|
||||
clearTimeout(timeoutId);
|
||||
resolve();
|
||||
}
|
||||
};
|
||||
worker.on('message', messageHandler);
|
||||
|
||||
// Timeout after specified milliseconds
|
||||
const timeoutId = setTimeout(() => {
|
||||
worker.removeListener('message', messageHandler);
|
||||
reject(new Error(`Worker failed to start within ${timeoutMs / 1000} seconds`));
|
||||
}, timeoutMs);
|
||||
});
|
||||
}
|
||||
|
||||
private forkWorker(serviceType: string): Worker {
|
||||
const worker = cluster.fork({ SERVICE_TYPE: serviceType });
|
||||
|
||||
@@ -116,7 +169,7 @@ class Application {
|
||||
if (worker) {
|
||||
const exitPromise = new Promise<void>((resolve) => {
|
||||
worker.once('exit', () => {
|
||||
Logger.info(`Worker ${worker.process.pid} exited`);
|
||||
Logger.info(`✌️ Worker ${worker.process.pid} exited`);
|
||||
resolve();
|
||||
});
|
||||
|
||||
@@ -124,7 +177,7 @@ class Application {
|
||||
worker.send('shutdown');
|
||||
} catch (error) {
|
||||
Logger.warn(
|
||||
`Failed to send shutdown to worker ${worker.process.pid}:`,
|
||||
`✌️ Failed to send shutdown to worker ${worker.process.pid}:`,
|
||||
error,
|
||||
);
|
||||
}
|
||||
@@ -139,14 +192,14 @@ class Application {
|
||||
Promise.all(workerPromises),
|
||||
new Promise<void>((resolve) => {
|
||||
setTimeout(() => {
|
||||
Logger.warn('Worker shutdown timeout reached');
|
||||
Logger.warn('✌️ Worker shutdown timeout reached');
|
||||
resolve();
|
||||
}, 10000);
|
||||
}),
|
||||
]);
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
Logger.error('Error during worker shutdown:', error);
|
||||
Logger.error('✌️ Error during worker shutdown:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
};
|
||||
@@ -158,7 +211,7 @@ class Application {
|
||||
private async startWorkerProcess() {
|
||||
const serviceType = process.env.SERVICE_TYPE;
|
||||
if (!serviceType || !['http', 'grpc'].includes(serviceType)) {
|
||||
Logger.error('Invalid SERVICE_TYPE:', serviceType);
|
||||
Logger.error('✌️ Invalid SERVICE_TYPE:', serviceType);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
@@ -173,7 +226,7 @@ class Application {
|
||||
|
||||
process.send?.('ready');
|
||||
} catch (error) {
|
||||
Logger.error(`${serviceType} worker failed:`, error);
|
||||
Logger.error(`✌️ ${serviceType} worker failed:`, error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
@@ -206,9 +259,20 @@ class Application {
|
||||
}
|
||||
|
||||
private setupWorkerShutdown(serviceType: string) {
|
||||
process.on('message', (msg) => {
|
||||
process.on('message', async (msg) => {
|
||||
if (msg === 'shutdown') {
|
||||
this.gracefulShutdown(serviceType);
|
||||
} else if (msg === 'reregister-crons' && serviceType === 'http') {
|
||||
// Re-register cron jobs when gRPC worker restarts
|
||||
try {
|
||||
Logger.info('✌️ Received reregister-crons message, re-registering cron jobs...');
|
||||
const CronService = (await import('./services/cron')).default;
|
||||
const cronService = Container.get(CronService);
|
||||
await cronService.autosave_crontab();
|
||||
Logger.info('✌️ Cron jobs re-registered successfully');
|
||||
} catch (error) {
|
||||
Logger.error('✌️ Failed to re-register cron jobs:', error);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
@@ -229,7 +293,7 @@ class Application {
|
||||
}
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
Logger.error(`[${serviceType}] Error during shutdown:`, error);
|
||||
Logger.error(`✌️ [${serviceType}] Error during shutdown:`, error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
@@ -237,6 +301,6 @@ class Application {
|
||||
|
||||
const app = new Application();
|
||||
app.start().catch((error) => {
|
||||
Logger.error('Application failed to start:', error);
|
||||
Logger.error('🙅♀️ Application failed to start:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
+4
-1
@@ -21,6 +21,7 @@ export class Crontab {
|
||||
extra_schedules?: Array<{ schedule: string }>;
|
||||
task_before?: string;
|
||||
task_after?: string;
|
||||
log_name?: string;
|
||||
|
||||
constructor(options: Crontab) {
|
||||
this.name = options.name;
|
||||
@@ -45,6 +46,7 @@ export class Crontab {
|
||||
this.extra_schedules = options.extra_schedules;
|
||||
this.task_before = options.task_before;
|
||||
this.task_after = options.task_after;
|
||||
this.log_name = options.log_name;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -55,7 +57,7 @@ export enum CrontabStatus {
|
||||
'disabled',
|
||||
}
|
||||
|
||||
export interface CronInstance extends Model<Crontab, Crontab>, Crontab { }
|
||||
export interface CronInstance extends Model<Crontab, Crontab>, Crontab {}
|
||||
export const CrontabModel = sequelize.define<CronInstance>('Crontab', {
|
||||
name: {
|
||||
unique: 'compositeIndex',
|
||||
@@ -84,4 +86,5 @@ export const CrontabModel = sequelize.define<CronInstance>('Crontab', {
|
||||
extra_schedules: DataTypes.JSON,
|
||||
task_before: DataTypes.STRING,
|
||||
task_after: DataTypes.STRING,
|
||||
log_name: DataTypes.STRING,
|
||||
});
|
||||
|
||||
+4
-1
@@ -1,5 +1,5 @@
|
||||
import { DataTypes, Model } from 'sequelize';
|
||||
import { sequelize } from '.';
|
||||
import { DataTypes, Model, ModelDefined } from 'sequelize';
|
||||
|
||||
export class Env {
|
||||
value?: string;
|
||||
@@ -9,6 +9,7 @@ export class Env {
|
||||
position?: number;
|
||||
name?: string;
|
||||
remarks?: string;
|
||||
isPinned?: 1 | 0;
|
||||
|
||||
constructor(options: Env) {
|
||||
this.value = options.value;
|
||||
@@ -21,6 +22,7 @@ export class Env {
|
||||
this.position = options.position;
|
||||
this.name = options.name;
|
||||
this.remarks = options.remarks || '';
|
||||
this.isPinned = options.isPinned || 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -42,4 +44,5 @@ export const EnvModel = sequelize.define<EnvInstance>('Env', {
|
||||
position: DataTypes.NUMBER,
|
||||
name: { type: DataTypes.STRING, unique: 'compositeIndex' },
|
||||
remarks: DataTypes.STRING,
|
||||
isPinned: DataTypes.NUMBER,
|
||||
});
|
||||
|
||||
@@ -56,6 +56,14 @@ export default async () => {
|
||||
try {
|
||||
await sequelize.query('alter table Crontabs add column task_after TEXT');
|
||||
} catch (error) {}
|
||||
try {
|
||||
await sequelize.query(
|
||||
'alter table Crontabs add column log_name VARCHAR(255)',
|
||||
);
|
||||
} catch (error) { }
|
||||
try {
|
||||
await sequelize.query('alter table Envs add column isPinned NUMBER');
|
||||
} catch (error) {}
|
||||
|
||||
Logger.info('✌️ DB loaded');
|
||||
} catch (error) {
|
||||
|
||||
+17
-5
@@ -1,8 +1,9 @@
|
||||
import path from 'path';
|
||||
import fs from 'fs/promises';
|
||||
import os from 'os';
|
||||
import chokidar from 'chokidar';
|
||||
import config from '../config/index';
|
||||
import { fileExist, promiseExec, rmPath } from '../config/util';
|
||||
import Logger from './logger';
|
||||
|
||||
async function linkToNodeModule(src: string, dst?: string) {
|
||||
const target = path.join(config.rootPath, 'node_modules', dst || src);
|
||||
@@ -17,8 +18,18 @@ async function linkToNodeModule(src: string, dst?: string) {
|
||||
}
|
||||
|
||||
async function linkCommand() {
|
||||
const commandPath = await promiseExec('which node');
|
||||
const commandDir = path.dirname(commandPath);
|
||||
const homeDir = os.homedir();
|
||||
let userBinDir = path.join(homeDir, 'bin');
|
||||
|
||||
try {
|
||||
await fs.mkdir(userBinDir, { recursive: true });
|
||||
await linkCommandToDir(userBinDir);
|
||||
} catch (error) {
|
||||
Logger.error('Linking command failed:', error);
|
||||
}
|
||||
}
|
||||
|
||||
async function linkCommandToDir(commandDir: string) {
|
||||
const linkShell = [
|
||||
{
|
||||
src: 'update.sh',
|
||||
@@ -42,6 +53,7 @@ async function linkCommand() {
|
||||
await fs.unlink(tmpTarget);
|
||||
}
|
||||
} catch (error) { }
|
||||
|
||||
await fs.symlink(source, tmpTarget);
|
||||
await fs.rename(tmpTarget, target);
|
||||
}
|
||||
@@ -58,6 +70,6 @@ export default async (src: string = 'deps') => {
|
||||
});
|
||||
|
||||
watcher
|
||||
.on('add', (path) => linkToNodeModule(src))
|
||||
.on('change', (path) => linkToNodeModule(src));
|
||||
.on('add', () => linkToNodeModule(src))
|
||||
.on('change', () => linkToNodeModule(src));
|
||||
};
|
||||
|
||||
@@ -97,6 +97,18 @@ message UpdateCronRequest {
|
||||
|
||||
message DeleteCronsRequest { repeated int32 ids = 1; }
|
||||
|
||||
message GetCronsRequest {
|
||||
optional string searchValue = 1;
|
||||
}
|
||||
|
||||
message GetCronByIdRequest { int32 id = 1; }
|
||||
|
||||
message EnableCronsRequest { repeated int32 ids = 1; }
|
||||
|
||||
message DisableCronsRequest { repeated int32 ids = 1; }
|
||||
|
||||
message RunCronsRequest { repeated int32 ids = 1; }
|
||||
|
||||
message CronsResponse {
|
||||
int32 code = 1;
|
||||
repeated CronItem data = 2;
|
||||
@@ -254,4 +266,9 @@ service Api {
|
||||
rpc CreateCron(CreateCronRequest) returns (CronResponse) {}
|
||||
rpc UpdateCron(UpdateCronRequest) returns (CronResponse) {}
|
||||
rpc DeleteCrons(DeleteCronsRequest) returns (Response) {}
|
||||
rpc GetCrons(GetCronsRequest) returns (CronsResponse) {}
|
||||
rpc GetCronById(GetCronByIdRequest) returns (CronResponse) {}
|
||||
rpc EnableCrons(EnableCronsRequest) returns (Response) {}
|
||||
rpc DisableCrons(DisableCronsRequest) returns (Response) {}
|
||||
rpc RunCrons(RunCronsRequest) returns (Response) {}
|
||||
}
|
||||
@@ -281,6 +281,26 @@ export interface DeleteCronsRequest {
|
||||
ids: number[];
|
||||
}
|
||||
|
||||
export interface GetCronsRequest {
|
||||
searchValue?: string | undefined;
|
||||
}
|
||||
|
||||
export interface GetCronByIdRequest {
|
||||
id: number;
|
||||
}
|
||||
|
||||
export interface EnableCronsRequest {
|
||||
ids: number[];
|
||||
}
|
||||
|
||||
export interface DisableCronsRequest {
|
||||
ids: number[];
|
||||
}
|
||||
|
||||
export interface RunCronsRequest {
|
||||
ids: number[];
|
||||
}
|
||||
|
||||
export interface CronsResponse {
|
||||
code: number;
|
||||
data: CronItem[];
|
||||
@@ -2207,6 +2227,332 @@ export const DeleteCronsRequest: MessageFns<DeleteCronsRequest> = {
|
||||
},
|
||||
};
|
||||
|
||||
function createBaseGetCronsRequest(): GetCronsRequest {
|
||||
return { searchValue: undefined };
|
||||
}
|
||||
|
||||
export const GetCronsRequest: MessageFns<GetCronsRequest> = {
|
||||
encode(message: GetCronsRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
|
||||
if (message.searchValue !== undefined) {
|
||||
writer.uint32(10).string(message.searchValue);
|
||||
}
|
||||
return writer;
|
||||
},
|
||||
|
||||
decode(input: BinaryReader | Uint8Array, length?: number): GetCronsRequest {
|
||||
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
|
||||
let end = length === undefined ? reader.len : reader.pos + length;
|
||||
const message = createBaseGetCronsRequest();
|
||||
while (reader.pos < end) {
|
||||
const tag = reader.uint32();
|
||||
switch (tag >>> 3) {
|
||||
case 1: {
|
||||
if (tag !== 10) {
|
||||
break;
|
||||
}
|
||||
|
||||
message.searchValue = reader.string();
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if ((tag & 7) === 4 || tag === 0) {
|
||||
break;
|
||||
}
|
||||
reader.skip(tag & 7);
|
||||
}
|
||||
return message;
|
||||
},
|
||||
|
||||
fromJSON(object: any): GetCronsRequest {
|
||||
return { searchValue: isSet(object.searchValue) ? globalThis.String(object.searchValue) : undefined };
|
||||
},
|
||||
|
||||
toJSON(message: GetCronsRequest): unknown {
|
||||
const obj: any = {};
|
||||
if (message.searchValue !== undefined) {
|
||||
obj.searchValue = message.searchValue;
|
||||
}
|
||||
return obj;
|
||||
},
|
||||
|
||||
create<I extends Exact<DeepPartial<GetCronsRequest>, I>>(base?: I): GetCronsRequest {
|
||||
return GetCronsRequest.fromPartial(base ?? ({} as any));
|
||||
},
|
||||
fromPartial<I extends Exact<DeepPartial<GetCronsRequest>, I>>(object: I): GetCronsRequest {
|
||||
const message = createBaseGetCronsRequest();
|
||||
message.searchValue = object.searchValue ?? undefined;
|
||||
return message;
|
||||
},
|
||||
};
|
||||
|
||||
function createBaseGetCronByIdRequest(): GetCronByIdRequest {
|
||||
return { id: 0 };
|
||||
}
|
||||
|
||||
export const GetCronByIdRequest: MessageFns<GetCronByIdRequest> = {
|
||||
encode(message: GetCronByIdRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
|
||||
if (message.id !== 0) {
|
||||
writer.uint32(8).int32(message.id);
|
||||
}
|
||||
return writer;
|
||||
},
|
||||
|
||||
decode(input: BinaryReader | Uint8Array, length?: number): GetCronByIdRequest {
|
||||
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
|
||||
let end = length === undefined ? reader.len : reader.pos + length;
|
||||
const message = createBaseGetCronByIdRequest();
|
||||
while (reader.pos < end) {
|
||||
const tag = reader.uint32();
|
||||
switch (tag >>> 3) {
|
||||
case 1: {
|
||||
if (tag !== 8) {
|
||||
break;
|
||||
}
|
||||
|
||||
message.id = reader.int32();
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if ((tag & 7) === 4 || tag === 0) {
|
||||
break;
|
||||
}
|
||||
reader.skip(tag & 7);
|
||||
}
|
||||
return message;
|
||||
},
|
||||
|
||||
fromJSON(object: any): GetCronByIdRequest {
|
||||
return { id: isSet(object.id) ? globalThis.Number(object.id) : 0 };
|
||||
},
|
||||
|
||||
toJSON(message: GetCronByIdRequest): unknown {
|
||||
const obj: any = {};
|
||||
if (message.id !== 0) {
|
||||
obj.id = Math.round(message.id);
|
||||
}
|
||||
return obj;
|
||||
},
|
||||
|
||||
create<I extends Exact<DeepPartial<GetCronByIdRequest>, I>>(base?: I): GetCronByIdRequest {
|
||||
return GetCronByIdRequest.fromPartial(base ?? ({} as any));
|
||||
},
|
||||
fromPartial<I extends Exact<DeepPartial<GetCronByIdRequest>, I>>(object: I): GetCronByIdRequest {
|
||||
const message = createBaseGetCronByIdRequest();
|
||||
message.id = object.id ?? 0;
|
||||
return message;
|
||||
},
|
||||
};
|
||||
|
||||
function createBaseEnableCronsRequest(): EnableCronsRequest {
|
||||
return { ids: [] };
|
||||
}
|
||||
|
||||
export const EnableCronsRequest: MessageFns<EnableCronsRequest> = {
|
||||
encode(message: EnableCronsRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
|
||||
writer.uint32(10).fork();
|
||||
for (const v of message.ids) {
|
||||
writer.int32(v);
|
||||
}
|
||||
writer.join();
|
||||
return writer;
|
||||
},
|
||||
|
||||
decode(input: BinaryReader | Uint8Array, length?: number): EnableCronsRequest {
|
||||
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
|
||||
let end = length === undefined ? reader.len : reader.pos + length;
|
||||
const message = createBaseEnableCronsRequest();
|
||||
while (reader.pos < end) {
|
||||
const tag = reader.uint32();
|
||||
switch (tag >>> 3) {
|
||||
case 1: {
|
||||
if (tag === 8) {
|
||||
message.ids.push(reader.int32());
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (tag === 10) {
|
||||
const end2 = reader.uint32() + reader.pos;
|
||||
while (reader.pos < end2) {
|
||||
message.ids.push(reader.int32());
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ((tag & 7) === 4 || tag === 0) {
|
||||
break;
|
||||
}
|
||||
reader.skip(tag & 7);
|
||||
}
|
||||
return message;
|
||||
},
|
||||
|
||||
fromJSON(object: any): EnableCronsRequest {
|
||||
return { ids: globalThis.Array.isArray(object?.ids) ? object.ids.map((e: any) => globalThis.Number(e)) : [] };
|
||||
},
|
||||
|
||||
toJSON(message: EnableCronsRequest): unknown {
|
||||
const obj: any = {};
|
||||
if (message.ids?.length) {
|
||||
obj.ids = message.ids.map((e) => Math.round(e));
|
||||
}
|
||||
return obj;
|
||||
},
|
||||
|
||||
create<I extends Exact<DeepPartial<EnableCronsRequest>, I>>(base?: I): EnableCronsRequest {
|
||||
return EnableCronsRequest.fromPartial(base ?? ({} as any));
|
||||
},
|
||||
fromPartial<I extends Exact<DeepPartial<EnableCronsRequest>, I>>(object: I): EnableCronsRequest {
|
||||
const message = createBaseEnableCronsRequest();
|
||||
message.ids = object.ids?.map((e) => e) || [];
|
||||
return message;
|
||||
},
|
||||
};
|
||||
|
||||
function createBaseDisableCronsRequest(): DisableCronsRequest {
|
||||
return { ids: [] };
|
||||
}
|
||||
|
||||
export const DisableCronsRequest: MessageFns<DisableCronsRequest> = {
|
||||
encode(message: DisableCronsRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
|
||||
writer.uint32(10).fork();
|
||||
for (const v of message.ids) {
|
||||
writer.int32(v);
|
||||
}
|
||||
writer.join();
|
||||
return writer;
|
||||
},
|
||||
|
||||
decode(input: BinaryReader | Uint8Array, length?: number): DisableCronsRequest {
|
||||
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
|
||||
let end = length === undefined ? reader.len : reader.pos + length;
|
||||
const message = createBaseDisableCronsRequest();
|
||||
while (reader.pos < end) {
|
||||
const tag = reader.uint32();
|
||||
switch (tag >>> 3) {
|
||||
case 1: {
|
||||
if (tag === 8) {
|
||||
message.ids.push(reader.int32());
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (tag === 10) {
|
||||
const end2 = reader.uint32() + reader.pos;
|
||||
while (reader.pos < end2) {
|
||||
message.ids.push(reader.int32());
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ((tag & 7) === 4 || tag === 0) {
|
||||
break;
|
||||
}
|
||||
reader.skip(tag & 7);
|
||||
}
|
||||
return message;
|
||||
},
|
||||
|
||||
fromJSON(object: any): DisableCronsRequest {
|
||||
return { ids: globalThis.Array.isArray(object?.ids) ? object.ids.map((e: any) => globalThis.Number(e)) : [] };
|
||||
},
|
||||
|
||||
toJSON(message: DisableCronsRequest): unknown {
|
||||
const obj: any = {};
|
||||
if (message.ids?.length) {
|
||||
obj.ids = message.ids.map((e) => Math.round(e));
|
||||
}
|
||||
return obj;
|
||||
},
|
||||
|
||||
create<I extends Exact<DeepPartial<DisableCronsRequest>, I>>(base?: I): DisableCronsRequest {
|
||||
return DisableCronsRequest.fromPartial(base ?? ({} as any));
|
||||
},
|
||||
fromPartial<I extends Exact<DeepPartial<DisableCronsRequest>, I>>(object: I): DisableCronsRequest {
|
||||
const message = createBaseDisableCronsRequest();
|
||||
message.ids = object.ids?.map((e) => e) || [];
|
||||
return message;
|
||||
},
|
||||
};
|
||||
|
||||
function createBaseRunCronsRequest(): RunCronsRequest {
|
||||
return { ids: [] };
|
||||
}
|
||||
|
||||
export const RunCronsRequest: MessageFns<RunCronsRequest> = {
|
||||
encode(message: RunCronsRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
|
||||
writer.uint32(10).fork();
|
||||
for (const v of message.ids) {
|
||||
writer.int32(v);
|
||||
}
|
||||
writer.join();
|
||||
return writer;
|
||||
},
|
||||
|
||||
decode(input: BinaryReader | Uint8Array, length?: number): RunCronsRequest {
|
||||
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
|
||||
let end = length === undefined ? reader.len : reader.pos + length;
|
||||
const message = createBaseRunCronsRequest();
|
||||
while (reader.pos < end) {
|
||||
const tag = reader.uint32();
|
||||
switch (tag >>> 3) {
|
||||
case 1: {
|
||||
if (tag === 8) {
|
||||
message.ids.push(reader.int32());
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (tag === 10) {
|
||||
const end2 = reader.uint32() + reader.pos;
|
||||
while (reader.pos < end2) {
|
||||
message.ids.push(reader.int32());
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ((tag & 7) === 4 || tag === 0) {
|
||||
break;
|
||||
}
|
||||
reader.skip(tag & 7);
|
||||
}
|
||||
return message;
|
||||
},
|
||||
|
||||
fromJSON(object: any): RunCronsRequest {
|
||||
return { ids: globalThis.Array.isArray(object?.ids) ? object.ids.map((e: any) => globalThis.Number(e)) : [] };
|
||||
},
|
||||
|
||||
toJSON(message: RunCronsRequest): unknown {
|
||||
const obj: any = {};
|
||||
if (message.ids?.length) {
|
||||
obj.ids = message.ids.map((e) => Math.round(e));
|
||||
}
|
||||
return obj;
|
||||
},
|
||||
|
||||
create<I extends Exact<DeepPartial<RunCronsRequest>, I>>(base?: I): RunCronsRequest {
|
||||
return RunCronsRequest.fromPartial(base ?? ({} as any));
|
||||
},
|
||||
fromPartial<I extends Exact<DeepPartial<RunCronsRequest>, I>>(object: I): RunCronsRequest {
|
||||
const message = createBaseRunCronsRequest();
|
||||
message.ids = object.ids?.map((e) => e) || [];
|
||||
return message;
|
||||
},
|
||||
};
|
||||
|
||||
function createBaseCronsResponse(): CronsResponse {
|
||||
return { code: 0, data: [], message: undefined };
|
||||
}
|
||||
@@ -3976,6 +4322,51 @@ export const ApiService = {
|
||||
responseSerialize: (value: Response) => Buffer.from(Response.encode(value).finish()),
|
||||
responseDeserialize: (value: Buffer) => Response.decode(value),
|
||||
},
|
||||
getCrons: {
|
||||
path: "/com.ql.api.Api/GetCrons",
|
||||
requestStream: false,
|
||||
responseStream: false,
|
||||
requestSerialize: (value: GetCronsRequest) => Buffer.from(GetCronsRequest.encode(value).finish()),
|
||||
requestDeserialize: (value: Buffer) => GetCronsRequest.decode(value),
|
||||
responseSerialize: (value: CronsResponse) => Buffer.from(CronsResponse.encode(value).finish()),
|
||||
responseDeserialize: (value: Buffer) => CronsResponse.decode(value),
|
||||
},
|
||||
getCronById: {
|
||||
path: "/com.ql.api.Api/GetCronById",
|
||||
requestStream: false,
|
||||
responseStream: false,
|
||||
requestSerialize: (value: GetCronByIdRequest) => Buffer.from(GetCronByIdRequest.encode(value).finish()),
|
||||
requestDeserialize: (value: Buffer) => GetCronByIdRequest.decode(value),
|
||||
responseSerialize: (value: CronResponse) => Buffer.from(CronResponse.encode(value).finish()),
|
||||
responseDeserialize: (value: Buffer) => CronResponse.decode(value),
|
||||
},
|
||||
enableCrons: {
|
||||
path: "/com.ql.api.Api/EnableCrons",
|
||||
requestStream: false,
|
||||
responseStream: false,
|
||||
requestSerialize: (value: EnableCronsRequest) => Buffer.from(EnableCronsRequest.encode(value).finish()),
|
||||
requestDeserialize: (value: Buffer) => EnableCronsRequest.decode(value),
|
||||
responseSerialize: (value: Response) => Buffer.from(Response.encode(value).finish()),
|
||||
responseDeserialize: (value: Buffer) => Response.decode(value),
|
||||
},
|
||||
disableCrons: {
|
||||
path: "/com.ql.api.Api/DisableCrons",
|
||||
requestStream: false,
|
||||
responseStream: false,
|
||||
requestSerialize: (value: DisableCronsRequest) => Buffer.from(DisableCronsRequest.encode(value).finish()),
|
||||
requestDeserialize: (value: Buffer) => DisableCronsRequest.decode(value),
|
||||
responseSerialize: (value: Response) => Buffer.from(Response.encode(value).finish()),
|
||||
responseDeserialize: (value: Buffer) => Response.decode(value),
|
||||
},
|
||||
runCrons: {
|
||||
path: "/com.ql.api.Api/RunCrons",
|
||||
requestStream: false,
|
||||
responseStream: false,
|
||||
requestSerialize: (value: RunCronsRequest) => Buffer.from(RunCronsRequest.encode(value).finish()),
|
||||
requestDeserialize: (value: Buffer) => RunCronsRequest.decode(value),
|
||||
responseSerialize: (value: Response) => Buffer.from(Response.encode(value).finish()),
|
||||
responseDeserialize: (value: Buffer) => Response.decode(value),
|
||||
},
|
||||
} as const;
|
||||
|
||||
export interface ApiServer extends UntypedServiceImplementation {
|
||||
@@ -3993,6 +4384,11 @@ export interface ApiServer extends UntypedServiceImplementation {
|
||||
createCron: handleUnaryCall<CreateCronRequest, CronResponse>;
|
||||
updateCron: handleUnaryCall<UpdateCronRequest, CronResponse>;
|
||||
deleteCrons: handleUnaryCall<DeleteCronsRequest, Response>;
|
||||
getCrons: handleUnaryCall<GetCronsRequest, CronsResponse>;
|
||||
getCronById: handleUnaryCall<GetCronByIdRequest, CronResponse>;
|
||||
enableCrons: handleUnaryCall<EnableCronsRequest, Response>;
|
||||
disableCrons: handleUnaryCall<DisableCronsRequest, Response>;
|
||||
runCrons: handleUnaryCall<RunCronsRequest, Response>;
|
||||
}
|
||||
|
||||
export interface ApiClient extends Client {
|
||||
@@ -4206,6 +4602,81 @@ export interface ApiClient extends Client {
|
||||
options: Partial<CallOptions>,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
getCrons(
|
||||
request: GetCronsRequest,
|
||||
callback: (error: ServiceError | null, response: CronsResponse) => void,
|
||||
): ClientUnaryCall;
|
||||
getCrons(
|
||||
request: GetCronsRequest,
|
||||
metadata: Metadata,
|
||||
callback: (error: ServiceError | null, response: CronsResponse) => void,
|
||||
): ClientUnaryCall;
|
||||
getCrons(
|
||||
request: GetCronsRequest,
|
||||
metadata: Metadata,
|
||||
options: Partial<CallOptions>,
|
||||
callback: (error: ServiceError | null, response: CronsResponse) => void,
|
||||
): ClientUnaryCall;
|
||||
getCronById(
|
||||
request: GetCronByIdRequest,
|
||||
callback: (error: ServiceError | null, response: CronResponse) => void,
|
||||
): ClientUnaryCall;
|
||||
getCronById(
|
||||
request: GetCronByIdRequest,
|
||||
metadata: Metadata,
|
||||
callback: (error: ServiceError | null, response: CronResponse) => void,
|
||||
): ClientUnaryCall;
|
||||
getCronById(
|
||||
request: GetCronByIdRequest,
|
||||
metadata: Metadata,
|
||||
options: Partial<CallOptions>,
|
||||
callback: (error: ServiceError | null, response: CronResponse) => void,
|
||||
): ClientUnaryCall;
|
||||
enableCrons(
|
||||
request: EnableCronsRequest,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
enableCrons(
|
||||
request: EnableCronsRequest,
|
||||
metadata: Metadata,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
enableCrons(
|
||||
request: EnableCronsRequest,
|
||||
metadata: Metadata,
|
||||
options: Partial<CallOptions>,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
disableCrons(
|
||||
request: DisableCronsRequest,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
disableCrons(
|
||||
request: DisableCronsRequest,
|
||||
metadata: Metadata,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
disableCrons(
|
||||
request: DisableCronsRequest,
|
||||
metadata: Metadata,
|
||||
options: Partial<CallOptions>,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
runCrons(
|
||||
request: RunCronsRequest,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
runCrons(
|
||||
request: RunCronsRequest,
|
||||
metadata: Metadata,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
runCrons(
|
||||
request: RunCronsRequest,
|
||||
metadata: Metadata,
|
||||
options: Partial<CallOptions>,
|
||||
callback: (error: ServiceError | null, response: Response) => void,
|
||||
): ClientUnaryCall;
|
||||
}
|
||||
|
||||
export const ApiClient = makeGenericClientConstructor(ApiService, "com.ql.api.Api") as unknown as {
|
||||
|
||||
@@ -30,6 +30,12 @@ import {
|
||||
UpdateCronRequest,
|
||||
DeleteCronsRequest,
|
||||
CronResponse,
|
||||
GetCronsRequest,
|
||||
CronsResponse,
|
||||
GetCronByIdRequest,
|
||||
EnableCronsRequest,
|
||||
DisableCronsRequest,
|
||||
RunCronsRequest,
|
||||
} from '../protos/api';
|
||||
import { NotificationInfo } from '../data/notify';
|
||||
|
||||
@@ -323,3 +329,116 @@ export const deleteCrons = async (
|
||||
callback(e);
|
||||
}
|
||||
};
|
||||
|
||||
export const getCrons = async (
|
||||
call: ServerUnaryCall<GetCronsRequest, CronsResponse>,
|
||||
callback: sendUnaryData<CronsResponse>,
|
||||
) => {
|
||||
try {
|
||||
const cronService = Container.get(CronService);
|
||||
const result = await cronService.crontabs({
|
||||
searchValue: call.request.searchValue || '',
|
||||
page: '0',
|
||||
size: '0',
|
||||
sorter: '',
|
||||
filters: '',
|
||||
queryString: '',
|
||||
});
|
||||
const data = result.data.map((x) => normalizeCronData(x as CronItem));
|
||||
callback(null, {
|
||||
code: 200,
|
||||
data: data.filter((x): x is CronItem => x !== undefined),
|
||||
});
|
||||
} catch (e: any) {
|
||||
callback(null, {
|
||||
code: 500,
|
||||
data: [],
|
||||
message: e.message,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
export const getCronById = async (
|
||||
call: ServerUnaryCall<GetCronByIdRequest, CronResponse>,
|
||||
callback: sendUnaryData<CronResponse>,
|
||||
) => {
|
||||
try {
|
||||
if (!call.request.id) {
|
||||
return callback(null, {
|
||||
code: 400,
|
||||
data: undefined,
|
||||
message: 'id parameter is required',
|
||||
});
|
||||
}
|
||||
|
||||
const cronService = Container.get(CronService);
|
||||
const data = (await cronService.getDb({ id: call.request.id })) as CronItem;
|
||||
callback(null, { code: 200, data: normalizeCronData(data) });
|
||||
} catch (e: any) {
|
||||
callback(null, {
|
||||
code: 404,
|
||||
data: undefined,
|
||||
message: e.message,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
export const enableCrons = async (
|
||||
call: ServerUnaryCall<EnableCronsRequest, Response>,
|
||||
callback: sendUnaryData<Response>,
|
||||
) => {
|
||||
try {
|
||||
if (!call.request.ids || call.request.ids.length === 0) {
|
||||
return callback(null, {
|
||||
code: 400,
|
||||
message: 'ids parameter is required',
|
||||
});
|
||||
}
|
||||
|
||||
const cronService = Container.get(CronService);
|
||||
await cronService.enabled(call.request.ids);
|
||||
callback(null, { code: 200 });
|
||||
} catch (e: any) {
|
||||
callback(e);
|
||||
}
|
||||
};
|
||||
|
||||
export const disableCrons = async (
|
||||
call: ServerUnaryCall<DisableCronsRequest, Response>,
|
||||
callback: sendUnaryData<Response>,
|
||||
) => {
|
||||
try {
|
||||
if (!call.request.ids || call.request.ids.length === 0) {
|
||||
return callback(null, {
|
||||
code: 400,
|
||||
message: 'ids parameter is required',
|
||||
});
|
||||
}
|
||||
|
||||
const cronService = Container.get(CronService);
|
||||
await cronService.disabled(call.request.ids);
|
||||
callback(null, { code: 200 });
|
||||
} catch (e: any) {
|
||||
callback(e);
|
||||
}
|
||||
};
|
||||
|
||||
export const runCrons = async (
|
||||
call: ServerUnaryCall<RunCronsRequest, Response>,
|
||||
callback: sendUnaryData<Response>,
|
||||
) => {
|
||||
try {
|
||||
if (!call.request.ids || call.request.ids.length === 0) {
|
||||
return callback(null, {
|
||||
code: 400,
|
||||
message: 'ids parameter is required',
|
||||
});
|
||||
}
|
||||
|
||||
const cronService = Container.get(CronService);
|
||||
await cronService.run(call.request.ids);
|
||||
callback(null, { code: 200 });
|
||||
} catch (e: any) {
|
||||
callback(e);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -17,14 +17,11 @@ const check = async (
|
||||
return callback(null, { status: 1 });
|
||||
}
|
||||
|
||||
const panelErrLog = await promiseExec(
|
||||
`tail -n 300 ~/.pm2/logs/panel-error.log`,
|
||||
);
|
||||
const scheduleErrLog = await promiseExec(
|
||||
`tail -n 300 ~/.pm2/logs/schedule-error.log`,
|
||||
const qinglongErrLog = await promiseExec(
|
||||
`tail -n 300 ~/.pm2/logs/qinglong-error.log`,
|
||||
);
|
||||
return callback(
|
||||
new Error(`${scheduleErrLog || ''}\n${panelErrLog || ''}\n${res}`.trim()),
|
||||
new Error(`${qinglongErrLog || ''}\n${res}`.trim()),
|
||||
);
|
||||
|
||||
default:
|
||||
|
||||
+50
-18
@@ -4,7 +4,7 @@ import config from '../config';
|
||||
import { Crontab, CrontabModel, CrontabStatus } from '../data/cron';
|
||||
import { exec, execSync } from 'child_process';
|
||||
import fs from 'fs/promises';
|
||||
import cron_parser from 'cron-parser';
|
||||
import { CronExpressionParser } from 'cron-parser';
|
||||
import {
|
||||
getFileContentByName,
|
||||
fileExist,
|
||||
@@ -24,10 +24,11 @@ import pickBy from 'lodash/pickBy';
|
||||
import omit from 'lodash/omit';
|
||||
import { writeFileWithLock } from '../shared/utils';
|
||||
import { ScheduleType } from '../interface/schedule';
|
||||
import { logStreamManager } from '../shared/logStreamManager';
|
||||
|
||||
@Service()
|
||||
export default class CronService {
|
||||
constructor(@Inject('logger') private logger: winston.Logger) {}
|
||||
constructor(@Inject('logger') private logger: winston.Logger) { }
|
||||
|
||||
private isNodeCron(cron: Crontab) {
|
||||
const { schedule, extra_schedules } = cron;
|
||||
@@ -49,9 +50,27 @@ export default class CronService {
|
||||
return this.isOnceSchedule(schedule) || this.isBootSchedule(schedule);
|
||||
}
|
||||
|
||||
private async getLogName(cron: Crontab) {
|
||||
const { log_name, command, id } = cron;
|
||||
if (log_name === '/dev/null') {
|
||||
return log_name;
|
||||
}
|
||||
let uniqPath = await getUniqPath(command, `${id}`);
|
||||
if (log_name) {
|
||||
const normalizedLogName = log_name.startsWith('/') ? log_name : path.join(config.logPath, log_name);
|
||||
if (normalizedLogName.startsWith(config.logPath)) {
|
||||
uniqPath = log_name;
|
||||
}
|
||||
}
|
||||
const logDirPath = path.resolve(config.logPath, `${uniqPath}`);
|
||||
await fs.mkdir(logDirPath, { recursive: true });
|
||||
return uniqPath;
|
||||
}
|
||||
|
||||
public async create(payload: Crontab): Promise<Crontab> {
|
||||
const tab = new Crontab(payload);
|
||||
tab.saved = false;
|
||||
tab.log_name = await this.getLogName(tab);
|
||||
const doc = await this.insert(tab);
|
||||
|
||||
if (isDemoEnv()) {
|
||||
@@ -82,6 +101,7 @@ export default class CronService {
|
||||
const doc = await this.getDb({ id: payload.id });
|
||||
const tab = new Crontab({ ...doc, ...payload });
|
||||
tab.saved = false;
|
||||
tab.log_name = await this.getLogName(tab);
|
||||
const newDoc = await this.updateDb(tab);
|
||||
|
||||
if (doc.isDisabled === 1 || isDemoEnv()) {
|
||||
@@ -142,7 +162,7 @@ export default class CronService {
|
||||
let cron;
|
||||
try {
|
||||
cron = await this.getDb({ id });
|
||||
} catch (err) {}
|
||||
} catch (err) { }
|
||||
if (!cron) {
|
||||
continue;
|
||||
}
|
||||
@@ -476,13 +496,12 @@ export default class CronService {
|
||||
`[panel][开始执行任务] 参数: ${JSON.stringify(params)}`,
|
||||
);
|
||||
|
||||
let { id, command, log_path } = cron;
|
||||
const uniqPath = await getUniqPath(command, `${id}`);
|
||||
let { id, command, log_name } = cron;
|
||||
|
||||
const uniqPath = log_name === '/dev/null' ? (await getUniqPath(command, `${id}`)) : log_name;
|
||||
const logTime = dayjs().format('YYYY-MM-DD-HH-mm-ss-SSS');
|
||||
const logDirPath = path.resolve(config.logPath, `${uniqPath}`);
|
||||
if (log_path?.split('/')?.every((x) => x !== uniqPath)) {
|
||||
await fs.mkdir(logDirPath, { recursive: true });
|
||||
}
|
||||
await fs.mkdir(logDirPath, { recursive: true });
|
||||
const logPath = `${uniqPath}/${logTime}.log`;
|
||||
const absolutePath = path.resolve(config.logPath, `${logPath}`);
|
||||
const cp = spawn(
|
||||
@@ -498,7 +517,7 @@ export default class CronService {
|
||||
{ where: { id } },
|
||||
);
|
||||
cp.stdout.on('data', async (data) => {
|
||||
await fs.appendFile(absolutePath, data.toString());
|
||||
await logStreamManager.write(absolutePath, data.toString());
|
||||
});
|
||||
cp.stderr.on('data', async (data) => {
|
||||
this.logger.info(
|
||||
@@ -506,7 +525,7 @@ export default class CronService {
|
||||
command,
|
||||
data.toString(),
|
||||
);
|
||||
await fs.appendFile(absolutePath, data.toString());
|
||||
await logStreamManager.write(absolutePath, data.toString());
|
||||
});
|
||||
cp.on('error', async (err) => {
|
||||
this.logger.error(
|
||||
@@ -514,7 +533,7 @@ export default class CronService {
|
||||
command,
|
||||
err,
|
||||
);
|
||||
await fs.appendFile(absolutePath, JSON.stringify(err));
|
||||
await logStreamManager.write(absolutePath, JSON.stringify(err));
|
||||
});
|
||||
|
||||
cp.on('exit', async (code) => {
|
||||
@@ -523,6 +542,8 @@ export default class CronService {
|
||||
JSON.stringify(params),
|
||||
code,
|
||||
);
|
||||
// Close the stream after task completion
|
||||
await logStreamManager.closeStream(absolutePath);
|
||||
await CrontabModel.update(
|
||||
{ status: CrontabStatus.idle, pid: undefined },
|
||||
{ where: { id } },
|
||||
@@ -564,7 +585,9 @@ export default class CronService {
|
||||
if (!doc) {
|
||||
return '';
|
||||
}
|
||||
|
||||
if (doc.log_name === '/dev/null') {
|
||||
return '日志设置为忽略';
|
||||
}
|
||||
const absolutePath = path.resolve(config.logPath, `${doc.log_path}`);
|
||||
const logFileExist = doc.log_path && (await fileExist(absolutePath));
|
||||
if (logFileExist) {
|
||||
@@ -607,9 +630,7 @@ export default class CronService {
|
||||
if (!command.startsWith(TASK_PREFIX) && !command.startsWith(QL_PREFIX)) {
|
||||
command = `${TASK_PREFIX}${tab.command}`;
|
||||
}
|
||||
let commandVariable = `real_time=${Boolean(realTime)} no_tee=true ID=${
|
||||
tab.id
|
||||
} `;
|
||||
let commandVariable = `real_time=${Boolean(realTime)} log_name=${tab.log_name} no_tee=true ID=${tab.id} `;
|
||||
if (tab.task_before) {
|
||||
commandVariable += `task_before='${tab.task_before
|
||||
.replace(/'/g, "'\\''")
|
||||
@@ -651,12 +672,23 @@ export default class CronService {
|
||||
|
||||
await writeFileWithLock(config.crontabFile, crontab_string);
|
||||
|
||||
execSync(`crontab ${config.crontabFile}`);
|
||||
try {
|
||||
execSync(`crontab ${config.crontabFile}`);
|
||||
} catch (error: any) {
|
||||
const errorMsg = error.message || String(error);
|
||||
this.logger.error('[crontab] Failed to update system crontab:', errorMsg);
|
||||
}
|
||||
|
||||
await CrontabModel.update({ saved: true }, { where: {} });
|
||||
}
|
||||
|
||||
public importCrontab() {
|
||||
exec('crontab -l', (error, stdout, stderr) => {
|
||||
exec('crontab -l', (error, stdout) => {
|
||||
if (error) {
|
||||
const errorMsg = error.message || String(error);
|
||||
this.logger.error('[crontab] Failed to read system crontab:', errorMsg);
|
||||
}
|
||||
|
||||
const lines = stdout.split('\n');
|
||||
const namePrefix = new Date().getTime();
|
||||
|
||||
@@ -670,7 +702,7 @@ export default class CronService {
|
||||
if (
|
||||
command &&
|
||||
schedule &&
|
||||
cron_parser.parseExpression(schedule).hasNext()
|
||||
CronExpressionParser.parse(schedule).hasNext()
|
||||
) {
|
||||
const name = namePrefix + '_' + index;
|
||||
|
||||
|
||||
+12
-4
@@ -1,7 +1,8 @@
|
||||
import { Service, Inject } from 'typedi';
|
||||
import groupBy from 'lodash/groupBy';
|
||||
import { FindOptions, Op } from 'sequelize';
|
||||
import { Inject, Service } from 'typedi';
|
||||
import winston from 'winston';
|
||||
import config from '../config';
|
||||
import * as fs from 'fs/promises';
|
||||
import {
|
||||
Env,
|
||||
EnvModel,
|
||||
@@ -11,8 +12,6 @@ import {
|
||||
minPosition,
|
||||
stepPosition,
|
||||
} from '../data/env';
|
||||
import groupBy from 'lodash/groupBy';
|
||||
import { FindOptions, Op } from 'sequelize';
|
||||
import { writeFileWithLock } from '../shared/utils';
|
||||
|
||||
@Service()
|
||||
@@ -147,6 +146,7 @@ export default class EnvService {
|
||||
}
|
||||
try {
|
||||
const result = await this.find(condition, [
|
||||
['isPinned', 'DESC'],
|
||||
['position', 'DESC'],
|
||||
['createdAt', 'ASC'],
|
||||
]);
|
||||
@@ -190,6 +190,14 @@ export default class EnvService {
|
||||
await this.set_envs();
|
||||
}
|
||||
|
||||
public async pin(ids: number[]) {
|
||||
await EnvModel.update({ isPinned: 1 }, { where: { id: ids } });
|
||||
}
|
||||
|
||||
public async unPin(ids: number[]) {
|
||||
await EnvModel.update({ isPinned: 0 }, { where: { id: ids } });
|
||||
}
|
||||
|
||||
public async set_envs() {
|
||||
const envs = await this.envs('', {
|
||||
name: { [Op.not]: null },
|
||||
|
||||
@@ -31,6 +31,7 @@ import { formatCommand, formatUrl } from '../config/subscription';
|
||||
import { CrontabModel } from '../data/cron';
|
||||
import CrontabService from './cron';
|
||||
import taskLimit from '../shared/pLimit';
|
||||
import { logStreamManager } from '../shared/logStreamManager';
|
||||
|
||||
@Service()
|
||||
export default class SubscriptionService {
|
||||
@@ -136,7 +137,7 @@ export default class SubscriptionService {
|
||||
let beforeStr = '';
|
||||
try {
|
||||
if (doc.sub_before) {
|
||||
await fs.appendFile(absolutePath, `\n## 执行before命令...\n\n`);
|
||||
await logStreamManager.write(absolutePath, `\n## 执行before命令...\n\n`);
|
||||
beforeStr = await promiseExec(doc.sub_before);
|
||||
}
|
||||
} catch (error: any) {
|
||||
@@ -144,7 +145,7 @@ export default class SubscriptionService {
|
||||
(error.stderr && error.stderr.toString()) || JSON.stringify(error);
|
||||
}
|
||||
if (beforeStr) {
|
||||
await fs.appendFile(absolutePath, `${beforeStr}\n`);
|
||||
await logStreamManager.write(absolutePath, `${beforeStr}\n`);
|
||||
}
|
||||
},
|
||||
onStart: async (cp: ChildProcessWithoutNullStreams, startTime) => {
|
||||
@@ -163,7 +164,7 @@ export default class SubscriptionService {
|
||||
let afterStr = '';
|
||||
try {
|
||||
if (sub.sub_after) {
|
||||
await fs.appendFile(absolutePath, `\n\n## 执行after命令...\n\n`);
|
||||
await logStreamManager.write(absolutePath, `\n\n## 执行after命令...\n\n`);
|
||||
afterStr = await promiseExec(sub.sub_after);
|
||||
}
|
||||
} catch (error: any) {
|
||||
@@ -171,16 +172,19 @@ export default class SubscriptionService {
|
||||
(error.stderr && error.stderr.toString()) || JSON.stringify(error);
|
||||
}
|
||||
if (afterStr) {
|
||||
await fs.appendFile(absolutePath, `${afterStr}\n`);
|
||||
await logStreamManager.write(absolutePath, `${afterStr}\n`);
|
||||
}
|
||||
|
||||
await fs.appendFile(
|
||||
await logStreamManager.write(
|
||||
absolutePath,
|
||||
`\n## 执行结束... ${endTime.format(
|
||||
'YYYY-MM-DD HH:mm:ss',
|
||||
)} 耗时 ${diff} 秒${LOG_END_SYMBOL}`,
|
||||
);
|
||||
|
||||
// Close the stream after task completion
|
||||
await logStreamManager.closeStream(absolutePath);
|
||||
|
||||
await SubscriptionModel.update(
|
||||
{ status: SubscriptionStatus.idle, pid: undefined },
|
||||
{ where: { id: sub.id } },
|
||||
@@ -195,12 +199,12 @@ export default class SubscriptionService {
|
||||
onError: async (message: string) => {
|
||||
const sub = await this.getDb({ id: doc.id });
|
||||
const absolutePath = await handleLogPath(sub.log_path as string);
|
||||
await fs.appendFile(absolutePath, `\n${message}`);
|
||||
await logStreamManager.write(absolutePath, `\n${message}`);
|
||||
},
|
||||
onLog: async (message: string) => {
|
||||
const sub = await this.getDb({ id: doc.id });
|
||||
const absolutePath = await handleLogPath(sub.log_path as string);
|
||||
await fs.appendFile(absolutePath, `\n${message}`);
|
||||
await logStreamManager.write(absolutePath, `\n${message}`);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
import { createWriteStream, WriteStream } from 'fs';
|
||||
import { EventEmitter } from 'events';
|
||||
|
||||
/**
|
||||
* Manages write streams for log files to improve performance by avoiding repeated file opens
|
||||
*/
|
||||
export class LogStreamManager extends EventEmitter {
|
||||
private streams: Map<string, WriteStream> = new Map();
|
||||
private pendingWrites: Map<string, Promise<void>> = new Map();
|
||||
|
||||
/**
|
||||
* Write data to a log file using a managed stream
|
||||
* @param filePath - Absolute path to the log file
|
||||
* @param data - Data to write to the log file
|
||||
*/
|
||||
async write(filePath: string, data: string): Promise<void> {
|
||||
// Wait for any pending writes to this file to complete
|
||||
const pending = this.pendingWrites.get(filePath);
|
||||
if (pending) {
|
||||
await pending;
|
||||
}
|
||||
|
||||
// Create a new promise for this write operation
|
||||
const writePromise = new Promise<void>((resolve, reject) => {
|
||||
let stream = this.streams.get(filePath);
|
||||
|
||||
if (!stream) {
|
||||
// Create a new write stream if one doesn't exist
|
||||
stream = createWriteStream(filePath, { flags: 'a' });
|
||||
this.streams.set(filePath, stream);
|
||||
|
||||
// Handle stream errors
|
||||
stream.on('error', (error) => {
|
||||
this.emit('error', { filePath, error });
|
||||
// Remove the stream from the map on error
|
||||
this.streams.delete(filePath);
|
||||
reject(error);
|
||||
});
|
||||
}
|
||||
|
||||
// Write the data
|
||||
const canContinue = stream.write(data, 'utf8', (error) => {
|
||||
if (error) {
|
||||
reject(error);
|
||||
} else {
|
||||
resolve();
|
||||
}
|
||||
});
|
||||
|
||||
// Handle backpressure
|
||||
if (!canContinue) {
|
||||
stream.once('drain', () => {
|
||||
// Stream is ready for more data
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
this.pendingWrites.set(filePath, writePromise);
|
||||
|
||||
try {
|
||||
await writePromise;
|
||||
} finally {
|
||||
this.pendingWrites.delete(filePath);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Close the stream for a specific file path
|
||||
* @param filePath - Absolute path to the log file
|
||||
*/
|
||||
async closeStream(filePath: string): Promise<void> {
|
||||
// Wait for any pending writes to complete
|
||||
const pending = this.pendingWrites.get(filePath);
|
||||
if (pending) {
|
||||
await pending.catch(() => {
|
||||
// Ignore errors on pending writes during close
|
||||
});
|
||||
}
|
||||
|
||||
const stream = this.streams.get(filePath);
|
||||
if (stream) {
|
||||
return new Promise<void>((resolve) => {
|
||||
stream.end(() => {
|
||||
this.streams.delete(filePath);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Close all open streams
|
||||
*/
|
||||
async closeAll(): Promise<void> {
|
||||
const closePromises = Array.from(this.streams.keys()).map((filePath) =>
|
||||
this.closeStream(filePath),
|
||||
);
|
||||
await Promise.all(closePromises);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the number of open streams
|
||||
*/
|
||||
getOpenStreamCount(): number {
|
||||
return this.streams.size;
|
||||
}
|
||||
}
|
||||
|
||||
// Export a singleton instance for shared use
|
||||
export const logStreamManager = new LogStreamManager();
|
||||
@@ -1,6 +1,8 @@
|
||||
import { Joi } from 'celebrate';
|
||||
import cron_parser from 'cron-parser';
|
||||
import { CronExpressionParser } from 'cron-parser';
|
||||
import { ScheduleType } from '../interface/schedule';
|
||||
import path from 'path';
|
||||
import config from '../config';
|
||||
|
||||
const validateSchedule = (value: string, helpers: any) => {
|
||||
if (
|
||||
@@ -11,7 +13,7 @@ const validateSchedule = (value: string, helpers: any) => {
|
||||
}
|
||||
|
||||
try {
|
||||
if (cron_parser.parseExpression(value).hasNext()) {
|
||||
if (CronExpressionParser.parse(value).hasNext()) {
|
||||
return value;
|
||||
}
|
||||
} catch (e) {
|
||||
@@ -37,4 +39,42 @@ export const commonCronSchema = {
|
||||
extra_schedules: Joi.array().optional().allow(null),
|
||||
task_before: Joi.string().optional().allow('').allow(null),
|
||||
task_after: Joi.string().optional().allow('').allow(null),
|
||||
log_name: Joi.string()
|
||||
.optional()
|
||||
.allow('')
|
||||
.allow(null)
|
||||
.custom((value, helpers) => {
|
||||
if (!value) return value;
|
||||
|
||||
// Check if it's an absolute path
|
||||
if (value.startsWith('/')) {
|
||||
// Allow /dev/null as special case
|
||||
if (value === '/dev/null') {
|
||||
return value;
|
||||
}
|
||||
|
||||
// For other absolute paths, ensure they are within the safe log directory
|
||||
const normalizedValue = path.normalize(value);
|
||||
const normalizedLogPath = path.normalize(config.logPath);
|
||||
|
||||
if (!normalizedValue.startsWith(normalizedLogPath)) {
|
||||
return helpers.error('string.unsafePath');
|
||||
}
|
||||
|
||||
return value;
|
||||
}
|
||||
|
||||
if (!/^(?!.*(?:^|\/)\.{1,2}(?:\/|$))(?:\/)?(?:[\w.-]+\/)*[\w.-]+\/?$/.test(value)) {
|
||||
return helpers.error('string.pattern.base');
|
||||
}
|
||||
if (value.length > 100) {
|
||||
return helpers.error('string.max');
|
||||
}
|
||||
return value;
|
||||
})
|
||||
.messages({
|
||||
'string.pattern.base': '日志名称只能包含字母、数字、下划线和连字符',
|
||||
'string.max': '日志名称不能超过100个字符',
|
||||
'string.unsafePath': '绝对路径必须在日志目录内或使用 /dev/null',
|
||||
}),
|
||||
};
|
||||
|
||||
@@ -1,22 +1,41 @@
|
||||
#!/bin/bash
|
||||
|
||||
export PATH="$HOME/bin:$PATH"
|
||||
|
||||
dir_shell=/ql/shell
|
||||
. $dir_shell/share.sh
|
||||
. $dir_shell/env.sh
|
||||
|
||||
export_ql_envs() {
|
||||
export BACK_PORT="${ql_port}"
|
||||
export GRPC_PORT="${ql_grpc_port}"
|
||||
}
|
||||
|
||||
log_with_style() {
|
||||
local level="$1"
|
||||
local message="$2"
|
||||
local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
|
||||
printf "\n[%s] [%7s] %s\n" "${timestamp}" "${level}" "${message}"
|
||||
}
|
||||
|
||||
# Fix DNS resolution issues in Alpine Linux
|
||||
# Alpine uses musl libc which has known DNS resolver issues with certain domains
|
||||
# Adding ndots:0 prevents unnecessary search domain appending
|
||||
if [ -f /etc/alpine-release ]; then
|
||||
if ! grep -q "^options ndots:0" /etc/resolv.conf 2>/dev/null; then
|
||||
echo "options ndots:0" >> /etc/resolv.conf
|
||||
log_with_style "INFO" "🔧 已配置 DNS 解析优化 (ndots:0)"
|
||||
fi
|
||||
fi
|
||||
|
||||
log_with_style "INFO" "🚀 1. 检测配置文件..."
|
||||
load_ql_envs
|
||||
export_ql_envs
|
||||
. $dir_shell/env.sh
|
||||
import_config "$@"
|
||||
fix_config
|
||||
|
||||
pm2 l &>/dev/null
|
||||
# Try to initialize PM2, but don't fail if it doesn't work
|
||||
pm2 l &>/dev/null || log_with_style "WARN" "PM2 初始化可能失败,将在启动时尝试使用备用方案"
|
||||
|
||||
log_with_style "INFO" "⚙️ 2. 启动 pm2 服务..."
|
||||
reload_pm2
|
||||
|
||||
+2
-1
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"private": true,
|
||||
"packageManager": "pnpm@8.3.1",
|
||||
"scripts": {
|
||||
"start": "concurrently -n w: npm:start:*",
|
||||
"start:back": "nodemon ./back/app.ts",
|
||||
@@ -61,7 +62,7 @@
|
||||
"celebrate": "^15.0.3",
|
||||
"chokidar": "^4.0.1",
|
||||
"cors": "^2.8.5",
|
||||
"cron-parser": "^4.9.0",
|
||||
"cron-parser": "^5.4.0",
|
||||
"cross-spawn": "^7.0.6",
|
||||
"dayjs": "^1.11.13",
|
||||
"dotenv": "^16.4.6",
|
||||
|
||||
Generated
+9977
-13085
File diff suppressed because it is too large
Load Diff
@@ -12,4 +12,32 @@ QLAPI.getEnvs({ searchValue: 'dddd' }).then((x) => {
|
||||
QLAPI.systemNotify({ title: '123', content: '231' }).then((x) => {
|
||||
console.log('systemNotify', x);
|
||||
});
|
||||
|
||||
// 查询定时任务 (Query cron tasks)
|
||||
QLAPI.getCrons({ searchValue: 'test' }).then((x) => {
|
||||
console.log('getCrons', x);
|
||||
});
|
||||
|
||||
// 通过ID查询定时任务 (Get cron by ID)
|
||||
QLAPI.getCronById({ id: 1 }).then((x) => {
|
||||
console.log('getCronById', x);
|
||||
}).catch((err) => {
|
||||
console.log('getCronById error', err);
|
||||
});
|
||||
|
||||
// 启用定时任务 (Enable cron tasks)
|
||||
QLAPI.enableCrons({ ids: [1, 2] }).then((x) => {
|
||||
console.log('enableCrons', x);
|
||||
});
|
||||
|
||||
// 禁用定时任务 (Disable cron tasks)
|
||||
QLAPI.disableCrons({ ids: [1, 2] }).then((x) => {
|
||||
console.log('disableCrons', x);
|
||||
});
|
||||
|
||||
// 手动执行定时任务 (Run cron tasks manually)
|
||||
QLAPI.runCrons({ ids: [1] }).then((x) => {
|
||||
console.log('runCrons', x);
|
||||
});
|
||||
|
||||
console.log('test desc');
|
||||
|
||||
+8
-8
@@ -41,7 +41,7 @@ add_cron_api() {
|
||||
fi
|
||||
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/crons?t=$currentTimeStamp" \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
--data-raw "{\"name\":\"${name//\"/\\\"}\",\"command\":\"${command//\"/\\\"}\",\"schedule\":\"$schedule\",\"sub_id\":$sub_id}" \
|
||||
@@ -71,7 +71,7 @@ update_cron_api() {
|
||||
fi
|
||||
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/crons?t=$currentTimeStamp" \
|
||||
-X 'PUT' \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
@@ -98,7 +98,7 @@ update_cron_command_api() {
|
||||
fi
|
||||
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/crons?t=$currentTimeStamp" \
|
||||
-X 'PUT' \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
@@ -118,7 +118,7 @@ del_cron_api() {
|
||||
local ids="$1"
|
||||
local currentTimeStamp=$(date +%s)
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/crons?t=$currentTimeStamp" \
|
||||
-X 'DELETE' \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
@@ -143,7 +143,7 @@ update_cron() {
|
||||
local runningTime="${6:-0}"
|
||||
local currentTimeStamp=$(date +%s)
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons/status?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/crons/status?t=$currentTimeStamp" \
|
||||
-X 'PUT' \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
@@ -165,7 +165,7 @@ notify_api() {
|
||||
local content="$2"
|
||||
local currentTimeStamp=$(date +%s)
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/system/notify?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/system/notify?t=$currentTimeStamp" \
|
||||
-X 'PUT' \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
@@ -185,7 +185,7 @@ find_cron_api() {
|
||||
local params="$1"
|
||||
local currentTimeStamp=$(date +%s)
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons/detail?$params&t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/crons/detail?$params&t=$currentTimeStamp" \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
--compressed
|
||||
@@ -204,7 +204,7 @@ update_auth_config() {
|
||||
local tip="$2"
|
||||
local currentTimeStamp=$(date +%s)
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/open/system/auth/reset?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/open/system/auth/reset?t=$currentTimeStamp" \
|
||||
-X 'PUT' \
|
||||
-H "Authorization: Bearer ${__ql_token__}" \
|
||||
-H "Content-Type: application/json;charset=UTF-8" \
|
||||
|
||||
+5
-5
@@ -24,14 +24,14 @@ copy_dep() {
|
||||
|
||||
pm2_log() {
|
||||
echo -e "---> pm2日志"
|
||||
local panelOut="/root/.pm2/logs/panel-out.log"
|
||||
local panelError="/root/.pm2/logs/panel-error.log"
|
||||
local panelOut="/root/.pm2/logs/qinglong-out.log"
|
||||
local panelError="/root/.pm2/logs/qinglong-error.log"
|
||||
tail -n 300 "$panelOut"
|
||||
tail -n 300 "$panelError"
|
||||
}
|
||||
|
||||
check_ql() {
|
||||
local api=$(curl -s --noproxy "*" "http://0.0.0.0:5700")
|
||||
local api=$(curl -s --noproxy "*" "http://0.0.0.0:${ql_port}")
|
||||
echo -e "\n=====> 检测面板\n\n$api\n"
|
||||
if [[ $api =~ "<div id=\"root\"></div>" ]]; then
|
||||
echo -e "=====> 面板服务启动正常\n"
|
||||
@@ -42,10 +42,10 @@ check_pm2() {
|
||||
pm2_log
|
||||
local currentTimeStamp=$(date +%s)
|
||||
local api=$(
|
||||
curl -s --noproxy "*" "http://0.0.0.0:5700/api/system?t=$currentTimeStamp" \
|
||||
curl -s --noproxy "*" "http://0.0.0.0:${ql_port}/api/system?t=$currentTimeStamp" \
|
||||
-H 'Accept: */*' \
|
||||
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36' \
|
||||
-H 'Referer: http://0.0.0.0:5700/crontab' \
|
||||
-H "Referer: http://0.0.0.0:${ql_port}/crontab" \
|
||||
-H 'Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7' \
|
||||
--compressed
|
||||
)
|
||||
|
||||
@@ -5,7 +5,7 @@ const { join } = require('path');
|
||||
class GrpcClient {
|
||||
static #config = {
|
||||
protoPath: join(process.env.QL_DIR, 'back/protos/api.proto'),
|
||||
serverAddress: '0.0.0.0:5500',
|
||||
serverAddress: `0.0.0.0:${process.env.GRPC_PORT || '5500'}`,
|
||||
protoOptions: {
|
||||
keepCase: true,
|
||||
longs: String,
|
||||
@@ -33,6 +33,11 @@ class GrpcClient {
|
||||
'createCron',
|
||||
'updateCron',
|
||||
'deleteCrons',
|
||||
'getCrons',
|
||||
'getCronById',
|
||||
'enableCrons',
|
||||
'disableCrons',
|
||||
'runCrons',
|
||||
];
|
||||
|
||||
#client;
|
||||
|
||||
@@ -0,0 +1,341 @@
|
||||
const Module = require('module');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
|
||||
// Get the QL_DIR and data directory paths
|
||||
const qlDir = process.env.QL_DIR || path.join(__dirname, '../../');
|
||||
let dataDir = process.env.QL_DATA_DIR || path.join(qlDir, 'data');
|
||||
|
||||
// Remove trailing slash if present
|
||||
dataDir = dataDir.replace(/\/$/, '');
|
||||
|
||||
// Normalize paths to avoid bypassing with relative paths or symlinks
|
||||
const normalizedQlDir = fs.existsSync(qlDir) ? fs.realpathSync(qlDir) : path.resolve(qlDir);
|
||||
const normalizedDataDir = fs.existsSync(dataDir) ? fs.realpathSync(dataDir) : path.resolve(dataDir);
|
||||
|
||||
// Protected directories - no write access allowed
|
||||
const protectedPaths = [
|
||||
path.join(normalizedQlDir, 'back'),
|
||||
path.join(normalizedQlDir, 'src'),
|
||||
path.join(normalizedQlDir, 'shell'),
|
||||
path.join(normalizedQlDir, 'sample'),
|
||||
path.join(normalizedQlDir, 'node_modules'),
|
||||
path.join(normalizedDataDir, 'config'),
|
||||
path.join(normalizedDataDir, 'db'),
|
||||
];
|
||||
|
||||
// Allowed write directories - scripts can write here
|
||||
const allowedWritePaths = [
|
||||
path.join(normalizedDataDir, 'scripts'),
|
||||
path.join(normalizedDataDir, 'log'),
|
||||
path.join(normalizedDataDir, 'repo'),
|
||||
path.join(normalizedDataDir, 'raw'),
|
||||
path.join(normalizedQlDir, '.tmp'),
|
||||
'/tmp',
|
||||
];
|
||||
|
||||
// Check if sandboxing is enabled (default: true)
|
||||
const sandboxEnabled = process.env.QL_DISABLE_SANDBOX !== 'true';
|
||||
|
||||
function isPathProtected(targetPath) {
|
||||
if (!sandboxEnabled) {
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
// Resolve to absolute path and follow symlinks
|
||||
const resolvedPath = fs.realpathSync.native ?
|
||||
fs.realpathSync.native(targetPath) :
|
||||
path.resolve(targetPath);
|
||||
|
||||
// Check if path is in a protected directory
|
||||
for (const protectedPath of protectedPaths) {
|
||||
if (resolvedPath.startsWith(protectedPath + path.sep) || resolvedPath === protectedPath) {
|
||||
// Check if it's in an allowed subdirectory
|
||||
const isInAllowedPath = allowedWritePaths.some(allowedPath =>
|
||||
resolvedPath.startsWith(allowedPath + path.sep) || resolvedPath === allowedPath
|
||||
);
|
||||
|
||||
if (!isInAllowedPath) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Also check if trying to write outside data/scripts without being in allowed paths
|
||||
const isInQlDir = resolvedPath.startsWith(normalizedQlDir + path.sep) || resolvedPath === normalizedQlDir;
|
||||
const isInDataDir = resolvedPath.startsWith(normalizedDataDir + path.sep) || resolvedPath === normalizedDataDir;
|
||||
|
||||
if (isInQlDir || isInDataDir) {
|
||||
const isInAllowedPath = allowedWritePaths.some(allowedPath =>
|
||||
resolvedPath.startsWith(allowedPath + path.sep) || resolvedPath === allowedPath
|
||||
);
|
||||
|
||||
if (!isInAllowedPath) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
} catch (err) {
|
||||
// If path doesn't exist yet, check parent directory
|
||||
const parentPath = path.dirname(targetPath);
|
||||
if (parentPath !== targetPath) {
|
||||
return isPathProtected(parentPath);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function createSecurityError(operation, targetPath) {
|
||||
const err = new Error(
|
||||
`Security Error: Script attempted to ${operation} protected path: ${targetPath}\n` +
|
||||
`Scripts are only allowed to write to: ${allowedWritePaths.join(', ')}`
|
||||
);
|
||||
err.code = 'EACCES';
|
||||
return err;
|
||||
}
|
||||
|
||||
// Store original fs methods
|
||||
const originalFS = {};
|
||||
const writeOperations = [
|
||||
'writeFile', 'writeFileSync',
|
||||
'appendFile', 'appendFileSync',
|
||||
'mkdir', 'mkdirSync',
|
||||
'rmdir', 'rmdirSync',
|
||||
'unlink', 'unlinkSync',
|
||||
'rm', 'rmSync',
|
||||
'rename', 'renameSync',
|
||||
'copyFile', 'copyFileSync',
|
||||
'chmod', 'chmodSync',
|
||||
'chown', 'chownSync',
|
||||
'link', 'linkSync',
|
||||
'symlink', 'symlinkSync',
|
||||
'truncate', 'truncateSync',
|
||||
'utimes', 'utimesSync',
|
||||
];
|
||||
|
||||
// Wrap fs methods
|
||||
for (const method of writeOperations) {
|
||||
if (fs[method]) {
|
||||
originalFS[method] = fs[method];
|
||||
}
|
||||
}
|
||||
|
||||
function wrapFsMethod(method, isSync) {
|
||||
return function(...args) {
|
||||
const targetPath = args[0];
|
||||
|
||||
if (isPathProtected(targetPath)) {
|
||||
const err = createSecurityError(method, targetPath);
|
||||
if (isSync) {
|
||||
throw err;
|
||||
} else {
|
||||
const callback = args[args.length - 1];
|
||||
if (typeof callback === 'function') {
|
||||
process.nextTick(() => callback(err));
|
||||
return;
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
// For rename/copy operations, check destination too
|
||||
if ((method.startsWith('rename') || method.startsWith('copy')) && args[1]) {
|
||||
if (isPathProtected(args[1])) {
|
||||
const err = createSecurityError(method, args[1]);
|
||||
if (isSync) {
|
||||
throw err;
|
||||
} else {
|
||||
const callback = args[args.length - 1];
|
||||
if (typeof callback === 'function') {
|
||||
process.nextTick(() => callback(err));
|
||||
return;
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return originalFS[method].apply(fs, args);
|
||||
};
|
||||
}
|
||||
|
||||
// Apply wrappers
|
||||
if (sandboxEnabled) {
|
||||
for (const method of writeOperations) {
|
||||
if (fs[method]) {
|
||||
const isSync = method.endsWith('Sync');
|
||||
fs[method] = wrapFsMethod(method, isSync);
|
||||
}
|
||||
}
|
||||
|
||||
// Wrap createWriteStream
|
||||
originalFS.createWriteStream = fs.createWriteStream;
|
||||
fs.createWriteStream = function(targetPath, options) {
|
||||
if (isPathProtected(targetPath)) {
|
||||
throw createSecurityError('createWriteStream', targetPath);
|
||||
}
|
||||
return originalFS.createWriteStream.call(fs, targetPath, options);
|
||||
};
|
||||
|
||||
// Wrap promises API if it exists
|
||||
if (fs.promises) {
|
||||
const promisesOriginal = {};
|
||||
const promisesMethods = [
|
||||
'writeFile', 'appendFile', 'mkdir', 'rmdir', 'unlink', 'rm',
|
||||
'rename', 'copyFile', 'chmod', 'chown', 'link', 'symlink',
|
||||
'truncate', 'utimes',
|
||||
];
|
||||
|
||||
for (const method of promisesMethods) {
|
||||
if (fs.promises[method]) {
|
||||
promisesOriginal[method] = fs.promises[method];
|
||||
fs.promises[method] = async function(...args) {
|
||||
const targetPath = args[0];
|
||||
|
||||
if (isPathProtected(targetPath)) {
|
||||
throw createSecurityError(method, targetPath);
|
||||
}
|
||||
|
||||
// For rename/copy operations, check destination too
|
||||
if ((method === 'rename' || method === 'copyFile') && args[1]) {
|
||||
if (isPathProtected(args[1])) {
|
||||
throw createSecurityError(method, args[1]);
|
||||
}
|
||||
}
|
||||
|
||||
return promisesOriginal[method].apply(fs.promises, args);
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Wrap child_process to prevent sandbox bypass via subprocesses
|
||||
let childProcessWrapped = false;
|
||||
if (sandboxEnabled) {
|
||||
// We need to get child_process before wrapping Module.prototype.require
|
||||
const childProcess = require('child_process');
|
||||
const originalSpawn = childProcess.spawn;
|
||||
const originalExec = childProcess.exec;
|
||||
const originalExecSync = childProcess.execSync;
|
||||
const originalExecFile = childProcess.execFile;
|
||||
const originalExecFileSync = childProcess.execFileSync;
|
||||
const originalFork = childProcess.fork;
|
||||
|
||||
// Helper to ensure NODE_OPTIONS and PYTHONPATH are set for subprocesses
|
||||
function ensureSandboxEnv(options = {}) {
|
||||
const env = { ...process.env, ...options.env };
|
||||
|
||||
// Ensure NODE_OPTIONS includes the sandbox
|
||||
const sandboxPreload = path.join(__dirname, 'sandbox.js');
|
||||
if (!env.NODE_OPTIONS) {
|
||||
env.NODE_OPTIONS = '';
|
||||
}
|
||||
if (!env.NODE_OPTIONS.includes(sandboxPreload)) {
|
||||
env.NODE_OPTIONS = `-r ${sandboxPreload} ${env.NODE_OPTIONS}`.trim();
|
||||
}
|
||||
|
||||
// Ensure PYTHONPATH includes the sandbox directory
|
||||
if (!env.PYTHONPATH) {
|
||||
env.PYTHONPATH = '';
|
||||
}
|
||||
if (!env.PYTHONPATH.includes(__dirname)) {
|
||||
env.PYTHONPATH = `${__dirname}:${env.PYTHONPATH}`;
|
||||
}
|
||||
|
||||
return { ...options, env };
|
||||
}
|
||||
|
||||
// Wrap spawn
|
||||
childProcess.spawn = function(...args) {
|
||||
if (args[2]) {
|
||||
args[2] = ensureSandboxEnv(args[2]);
|
||||
} else if (args.length >= 3) {
|
||||
args[2] = ensureSandboxEnv({});
|
||||
}
|
||||
return originalSpawn.apply(childProcess, args);
|
||||
};
|
||||
|
||||
// Wrap exec
|
||||
childProcess.exec = function(...args) {
|
||||
const callback = typeof args[args.length - 1] === 'function' ? args[args.length - 1] : undefined;
|
||||
const optionsIndex = callback ? args.length - 2 : args.length - 1;
|
||||
|
||||
if (args[optionsIndex] && typeof args[optionsIndex] === 'object') {
|
||||
args[optionsIndex] = ensureSandboxEnv(args[optionsIndex]);
|
||||
} else if (optionsIndex > 0) {
|
||||
args.splice(optionsIndex, 0, ensureSandboxEnv({}));
|
||||
}
|
||||
return originalExec.apply(childProcess, args);
|
||||
};
|
||||
|
||||
// Wrap execSync
|
||||
childProcess.execSync = function(...args) {
|
||||
if (args[1]) {
|
||||
args[1] = ensureSandboxEnv(args[1]);
|
||||
} else {
|
||||
args[1] = ensureSandboxEnv({});
|
||||
}
|
||||
return originalExecSync.apply(childProcess, args);
|
||||
};
|
||||
|
||||
// Wrap execFile
|
||||
childProcess.execFile = function(...args) {
|
||||
const callback = typeof args[args.length - 1] === 'function' ? args[args.length - 1] : undefined;
|
||||
const optionsIndex = callback ? args.length - 2 : args.length - 1;
|
||||
|
||||
if (args[optionsIndex] && typeof args[optionsIndex] === 'object') {
|
||||
args[optionsIndex] = ensureSandboxEnv(args[optionsIndex]);
|
||||
} else if (optionsIndex > 1) {
|
||||
args.splice(optionsIndex, 0, ensureSandboxEnv({}));
|
||||
}
|
||||
return originalExecFile.apply(childProcess, args);
|
||||
};
|
||||
|
||||
// Wrap execFileSync
|
||||
childProcess.execFileSync = function(...args) {
|
||||
if (args[2]) {
|
||||
args[2] = ensureSandboxEnv(args[2]);
|
||||
} else if (args.length >= 3) {
|
||||
args[2] = ensureSandboxEnv({});
|
||||
}
|
||||
return originalExecFileSync.apply(childProcess, args);
|
||||
};
|
||||
|
||||
// Wrap fork
|
||||
childProcess.fork = function(...args) {
|
||||
if (args[2]) {
|
||||
args[2] = ensureSandboxEnv(args[2]);
|
||||
} else if (args.length >= 3) {
|
||||
args[2] = ensureSandboxEnv({});
|
||||
}
|
||||
return originalFork.apply(childProcess, args);
|
||||
};
|
||||
|
||||
childProcessWrapped = true;
|
||||
}
|
||||
|
||||
// Prevent requiring the original fs or child_process modules to bypass sandbox
|
||||
const originalRequire = Module.prototype.require;
|
||||
Module.prototype.require = function(id) {
|
||||
const module = originalRequire.apply(this, arguments);
|
||||
|
||||
// Return wrapped fs module
|
||||
if (id === 'fs' || id === 'node:fs') {
|
||||
return fs;
|
||||
}
|
||||
|
||||
// For child_process, we already wrapped it above, so just return it
|
||||
// (no need to re-require as that would cause recursion)
|
||||
|
||||
return module;
|
||||
};
|
||||
|
||||
module.exports = {
|
||||
sandboxEnabled,
|
||||
isPathProtected,
|
||||
protectedPaths,
|
||||
allowedWritePaths,
|
||||
};
|
||||
@@ -0,0 +1,408 @@
|
||||
import os
|
||||
import sys
|
||||
import builtins
|
||||
from pathlib import Path
|
||||
|
||||
# Get the QL_DIR and data directory paths
|
||||
ql_dir = os.environ.get('QL_DIR', os.path.join(os.path.dirname(__file__), '../..'))
|
||||
data_dir = os.environ.get('QL_DATA_DIR', os.path.join(ql_dir, 'data'))
|
||||
|
||||
# Normalize paths to avoid bypassing with relative paths or symlinks
|
||||
try:
|
||||
normalized_ql_dir = os.path.realpath(ql_dir)
|
||||
normalized_data_dir = os.path.realpath(data_dir)
|
||||
except:
|
||||
normalized_ql_dir = os.path.abspath(ql_dir)
|
||||
normalized_data_dir = os.path.abspath(data_dir)
|
||||
|
||||
# Protected directories - no write access allowed
|
||||
protected_paths = [
|
||||
os.path.join(normalized_ql_dir, 'back'),
|
||||
os.path.join(normalized_ql_dir, 'src'),
|
||||
os.path.join(normalized_ql_dir, 'shell'),
|
||||
os.path.join(normalized_ql_dir, 'sample'),
|
||||
os.path.join(normalized_ql_dir, 'node_modules'),
|
||||
os.path.join(normalized_data_dir, 'config'),
|
||||
os.path.join(normalized_data_dir, 'db'),
|
||||
]
|
||||
|
||||
# Allowed write directories - scripts can write here
|
||||
allowed_write_paths = [
|
||||
os.path.join(normalized_data_dir, 'scripts'),
|
||||
os.path.join(normalized_data_dir, 'log'),
|
||||
os.path.join(normalized_data_dir, 'repo'),
|
||||
os.path.join(normalized_data_dir, 'raw'),
|
||||
os.path.join(normalized_ql_dir, '.tmp'),
|
||||
'/tmp',
|
||||
]
|
||||
|
||||
# Check if sandboxing is enabled (default: true)
|
||||
sandbox_enabled = os.environ.get('QL_DISABLE_SANDBOX') != 'true'
|
||||
|
||||
def is_path_protected(target_path):
|
||||
"""Check if a path is protected from write operations"""
|
||||
if not sandbox_enabled:
|
||||
return False
|
||||
|
||||
try:
|
||||
# Resolve to absolute path and follow symlinks
|
||||
resolved_path = os.path.realpath(target_path)
|
||||
|
||||
# Check if path is in a protected directory
|
||||
for protected_path in protected_paths:
|
||||
if resolved_path.startswith(protected_path + os.sep) or resolved_path == protected_path:
|
||||
# Check if it's in an allowed subdirectory
|
||||
is_in_allowed_path = any(
|
||||
resolved_path.startswith(allowed_path + os.sep) or resolved_path == allowed_path
|
||||
for allowed_path in allowed_write_paths
|
||||
)
|
||||
|
||||
if not is_in_allowed_path:
|
||||
return True
|
||||
|
||||
# Also check if trying to write inside ql_dir or data_dir without being in allowed paths
|
||||
is_in_ql_dir = resolved_path.startswith(normalized_ql_dir + os.sep) or resolved_path == normalized_ql_dir
|
||||
is_in_data_dir = resolved_path.startswith(normalized_data_dir + os.sep) or resolved_path == normalized_data_dir
|
||||
|
||||
if is_in_ql_dir or is_in_data_dir:
|
||||
is_in_allowed_path = any(
|
||||
resolved_path.startswith(allowed_path + os.sep) or resolved_path == allowed_path
|
||||
for allowed_path in allowed_write_paths
|
||||
)
|
||||
|
||||
if not is_in_allowed_path:
|
||||
return True
|
||||
|
||||
return False
|
||||
except:
|
||||
# If path doesn't exist yet, check parent directory
|
||||
parent_path = os.path.dirname(target_path)
|
||||
if parent_path != target_path:
|
||||
return is_path_protected(parent_path)
|
||||
return False
|
||||
|
||||
def create_security_error(operation, target_path):
|
||||
"""Create a security error for unauthorized file operations"""
|
||||
return PermissionError(
|
||||
f"Security Error: Script attempted to {operation} protected path: {target_path}\n"
|
||||
f"Scripts are only allowed to write to: {', '.join(allowed_write_paths)}"
|
||||
)
|
||||
|
||||
# Store original functions
|
||||
original_open = builtins.open
|
||||
original_os_remove = os.remove
|
||||
original_os_unlink = os.unlink
|
||||
original_os_rmdir = os.rmdir
|
||||
original_os_mkdir = os.mkdir
|
||||
original_os_makedirs = os.makedirs
|
||||
original_os_rename = os.rename
|
||||
original_os_replace = os.replace
|
||||
original_os_chmod = os.chmod
|
||||
original_os_chown = os.chown if hasattr(os, 'chown') else None
|
||||
original_os_link = os.link if hasattr(os, 'link') else None
|
||||
original_os_symlink = os.symlink if hasattr(os, 'symlink') else None
|
||||
original_os_truncate = os.truncate if hasattr(os, 'truncate') else None
|
||||
original_os_utime = os.utime if hasattr(os, 'utime') else None
|
||||
|
||||
# Wrap open() to check write operations
|
||||
def sandboxed_open(file, mode='r', *args, **kwargs):
|
||||
"""Wrapped open() that checks for protected paths on write operations"""
|
||||
if sandbox_enabled and isinstance(mode, str) and any(m in mode for m in ['w', 'a', 'x', '+']):
|
||||
if is_path_protected(file):
|
||||
raise create_security_error('open for writing', file)
|
||||
return original_open(file, mode, *args, **kwargs)
|
||||
|
||||
# Wrap os functions
|
||||
def sandboxed_remove(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('remove', path)
|
||||
return original_os_remove(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_unlink(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('unlink', path)
|
||||
return original_os_unlink(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_rmdir(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('rmdir', path)
|
||||
return original_os_rmdir(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_mkdir(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('mkdir', path)
|
||||
return original_os_mkdir(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_makedirs(name, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(name):
|
||||
raise create_security_error('makedirs', name)
|
||||
return original_os_makedirs(name, *args, **kwargs)
|
||||
|
||||
def sandboxed_rename(src, dst, *args, **kwargs):
|
||||
if sandbox_enabled:
|
||||
if is_path_protected(src):
|
||||
raise create_security_error('rename (source)', src)
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('rename (destination)', dst)
|
||||
return original_os_rename(src, dst, *args, **kwargs)
|
||||
|
||||
def sandboxed_replace(src, dst, *args, **kwargs):
|
||||
if sandbox_enabled:
|
||||
if is_path_protected(src):
|
||||
raise create_security_error('replace (source)', src)
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('replace (destination)', dst)
|
||||
return original_os_replace(src, dst, *args, **kwargs)
|
||||
|
||||
def sandboxed_chmod(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('chmod', path)
|
||||
return original_os_chmod(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_chown(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('chown', path)
|
||||
return original_os_chown(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_link(src, dst, *args, **kwargs):
|
||||
if sandbox_enabled:
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('link', dst)
|
||||
return original_os_link(src, dst, *args, **kwargs)
|
||||
|
||||
def sandboxed_symlink(src, dst, *args, **kwargs):
|
||||
if sandbox_enabled:
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('symlink', dst)
|
||||
return original_os_symlink(src, dst, *args, **kwargs)
|
||||
|
||||
def sandboxed_truncate(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('truncate', path)
|
||||
return original_os_truncate(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_utime(path, *args, **kwargs):
|
||||
if sandbox_enabled and is_path_protected(path):
|
||||
raise create_security_error('utime', path)
|
||||
return original_os_utime(path, *args, **kwargs)
|
||||
|
||||
# Apply sandbox wrappers
|
||||
if sandbox_enabled:
|
||||
builtins.open = sandboxed_open
|
||||
os.remove = sandboxed_remove
|
||||
os.unlink = sandboxed_unlink
|
||||
os.rmdir = sandboxed_rmdir
|
||||
os.mkdir = sandboxed_mkdir
|
||||
os.makedirs = sandboxed_makedirs
|
||||
os.rename = sandboxed_rename
|
||||
os.replace = sandboxed_replace
|
||||
os.chmod = sandboxed_chmod
|
||||
if original_os_chown:
|
||||
os.chown = sandboxed_chown
|
||||
if original_os_link:
|
||||
os.link = sandboxed_link
|
||||
if original_os_symlink:
|
||||
os.symlink = sandboxed_symlink
|
||||
if original_os_truncate:
|
||||
os.truncate = sandboxed_truncate
|
||||
if original_os_utime:
|
||||
os.utime = sandboxed_utime
|
||||
|
||||
# Wrap shutil if it's imported
|
||||
try:
|
||||
import shutil
|
||||
original_shutil_rmtree = shutil.rmtree
|
||||
original_shutil_copy = shutil.copy
|
||||
original_shutil_copy2 = shutil.copy2
|
||||
original_shutil_copytree = shutil.copytree
|
||||
original_shutil_move = shutil.move
|
||||
|
||||
def sandboxed_rmtree(path, *args, **kwargs):
|
||||
if is_path_protected(path):
|
||||
raise create_security_error('rmtree', path)
|
||||
return original_shutil_rmtree(path, *args, **kwargs)
|
||||
|
||||
def sandboxed_copy(src, dst, *args, **kwargs):
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('copy', dst)
|
||||
return original_shutil_copy(src, dst, *args, **kwargs)
|
||||
|
||||
def sandboxed_copy2(src, dst, *args, **kwargs):
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('copy2', dst)
|
||||
return original_shutil_copy2(src, dst, *args, **kwargs)
|
||||
|
||||
def sandboxed_copytree(src, dst, *args, **kwargs):
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('copytree', dst)
|
||||
return original_shutil_copytree(src, dst, *args, **kwargs)
|
||||
|
||||
def sandboxed_move(src, dst, *args, **kwargs):
|
||||
if is_path_protected(src):
|
||||
raise create_security_error('move (source)', src)
|
||||
if is_path_protected(dst):
|
||||
raise create_security_error('move (destination)', dst)
|
||||
return original_shutil_move(src, dst, *args, **kwargs)
|
||||
|
||||
shutil.rmtree = sandboxed_rmtree
|
||||
shutil.copy = sandboxed_copy
|
||||
shutil.copy2 = sandboxed_copy2
|
||||
shutil.copytree = sandboxed_copytree
|
||||
shutil.move = sandboxed_move
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# Wrap pathlib.Path if available
|
||||
try:
|
||||
original_path_write_text = Path.write_text
|
||||
original_path_write_bytes = Path.write_bytes
|
||||
original_path_touch = Path.touch
|
||||
original_path_mkdir = Path.mkdir
|
||||
original_path_rmdir = Path.rmdir
|
||||
original_path_unlink = Path.unlink
|
||||
original_path_rename = Path.rename
|
||||
original_path_replace = Path.replace
|
||||
original_path_chmod = Path.chmod
|
||||
|
||||
def sandboxed_path_write_text(self, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.write_text', str(self))
|
||||
return original_path_write_text(self, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_write_bytes(self, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.write_bytes', str(self))
|
||||
return original_path_write_bytes(self, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_touch(self, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.touch', str(self))
|
||||
return original_path_touch(self, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_mkdir(self, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.mkdir', str(self))
|
||||
return original_path_mkdir(self, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_rmdir(self, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.rmdir', str(self))
|
||||
return original_path_rmdir(self, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_unlink(self, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.unlink', str(self))
|
||||
return original_path_unlink(self, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_rename(self, target, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.rename (source)', str(self))
|
||||
if is_path_protected(str(target)):
|
||||
raise create_security_error('Path.rename (target)', str(target))
|
||||
return original_path_rename(self, target, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_replace(self, target, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.replace (source)', str(self))
|
||||
if is_path_protected(str(target)):
|
||||
raise create_security_error('Path.replace (target)', str(target))
|
||||
return original_path_replace(self, target, *args, **kwargs)
|
||||
|
||||
def sandboxed_path_chmod(self, *args, **kwargs):
|
||||
if is_path_protected(str(self)):
|
||||
raise create_security_error('Path.chmod', str(self))
|
||||
return original_path_chmod(self, *args, **kwargs)
|
||||
|
||||
Path.write_text = sandboxed_path_write_text
|
||||
Path.write_bytes = sandboxed_path_write_bytes
|
||||
Path.touch = sandboxed_path_touch
|
||||
Path.mkdir = sandboxed_path_mkdir
|
||||
Path.rmdir = sandboxed_path_rmdir
|
||||
Path.unlink = sandboxed_path_unlink
|
||||
Path.rename = sandboxed_path_rename
|
||||
Path.replace = sandboxed_path_replace
|
||||
Path.chmod = sandboxed_path_chmod
|
||||
except:
|
||||
pass
|
||||
|
||||
# Wrap subprocess to prevent sandbox bypass via subprocesses
|
||||
try:
|
||||
import subprocess
|
||||
|
||||
# Helper to ensure PYTHONPATH is set for subprocesses
|
||||
def ensure_sandbox_env(env=None):
|
||||
if env is None:
|
||||
env = os.environ.copy()
|
||||
else:
|
||||
env = env.copy()
|
||||
|
||||
# Ensure PYTHONPATH includes the sandbox directory
|
||||
sandbox_dir = os.path.dirname(__file__)
|
||||
if 'PYTHONPATH' not in env:
|
||||
env['PYTHONPATH'] = ''
|
||||
if sandbox_dir not in env['PYTHONPATH']:
|
||||
env['PYTHONPATH'] = f"{sandbox_dir}:{env['PYTHONPATH']}"
|
||||
|
||||
return env
|
||||
|
||||
# Store original functions
|
||||
original_popen = subprocess.Popen
|
||||
original_run = subprocess.run
|
||||
original_call = subprocess.call
|
||||
original_check_call = subprocess.check_call
|
||||
original_check_output = subprocess.check_output
|
||||
|
||||
# Wrap Popen
|
||||
class SandboxedPopen(subprocess.Popen):
|
||||
def __init__(self, *args, **kwargs):
|
||||
if 'env' in kwargs:
|
||||
kwargs['env'] = ensure_sandbox_env(kwargs['env'])
|
||||
else:
|
||||
kwargs['env'] = ensure_sandbox_env()
|
||||
original_popen.__init__(self, *args, **kwargs)
|
||||
|
||||
subprocess.Popen = SandboxedPopen
|
||||
|
||||
# Wrap run
|
||||
def sandboxed_run(*args, **kwargs):
|
||||
if 'env' in kwargs:
|
||||
kwargs['env'] = ensure_sandbox_env(kwargs['env'])
|
||||
else:
|
||||
kwargs['env'] = ensure_sandbox_env()
|
||||
return original_run(*args, **kwargs)
|
||||
|
||||
subprocess.run = sandboxed_run
|
||||
|
||||
# Wrap call
|
||||
def sandboxed_call(*args, **kwargs):
|
||||
if 'env' in kwargs:
|
||||
kwargs['env'] = ensure_sandbox_env(kwargs['env'])
|
||||
else:
|
||||
kwargs['env'] = ensure_sandbox_env()
|
||||
return original_call(*args, **kwargs)
|
||||
|
||||
subprocess.call = sandboxed_call
|
||||
|
||||
# Wrap check_call
|
||||
def sandboxed_check_call(*args, **kwargs):
|
||||
if 'env' in kwargs:
|
||||
kwargs['env'] = ensure_sandbox_env(kwargs['env'])
|
||||
else:
|
||||
kwargs['env'] = ensure_sandbox_env()
|
||||
return original_check_call(*args, **kwargs)
|
||||
|
||||
subprocess.check_call = sandboxed_check_call
|
||||
|
||||
# Wrap check_output
|
||||
def sandboxed_check_output(*args, **kwargs):
|
||||
if 'env' in kwargs:
|
||||
kwargs['env'] = ensure_sandbox_env(kwargs['env'])
|
||||
else:
|
||||
kwargs['env'] = ensure_sandbox_env()
|
||||
return original_check_output(*args, **kwargs)
|
||||
|
||||
subprocess.check_output = sandboxed_check_output
|
||||
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
// Load sandbox first to protect filesystem
|
||||
require('./sandbox.js');
|
||||
|
||||
const { execSync } = require('child_process');
|
||||
const client = require('./client.js');
|
||||
require(`./env.js`);
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
# Load sandbox first to protect filesystem
|
||||
import sandbox
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
+32
-11
@@ -59,15 +59,10 @@ list_own_user=$dir_list_tmp/own_user.list
|
||||
list_own_add=$dir_list_tmp/own_add.list
|
||||
list_own_drop=$dir_list_tmp/own_drop.list
|
||||
|
||||
## 软连接及其原始文件对应关系
|
||||
link_name=(
|
||||
task
|
||||
ql
|
||||
)
|
||||
original_name=(
|
||||
task.sh
|
||||
update.sh
|
||||
)
|
||||
|
||||
init_env() {
|
||||
local pnpm_global_path=$(pnpm root -g 2>/dev/null)
|
||||
@@ -84,15 +79,20 @@ init_env() {
|
||||
export PYTHONUNBUFFERED=1
|
||||
}
|
||||
|
||||
load_ql_envs() {
|
||||
ql_base_url=${QlBaseUrl:-"/"}
|
||||
ql_port=${QlPort:-"5700"}
|
||||
ql_grpc_port=${QlGrpcPort:-"5500"}
|
||||
current_branch=${QL_BRANCH:-""}
|
||||
}
|
||||
|
||||
import_config() {
|
||||
[[ -f $file_config_user ]] && . $file_config_user
|
||||
|
||||
ql_base_url=${QlBaseUrl:-"/"}
|
||||
ql_port=${QlPort:-"5700"}
|
||||
load_ql_envs
|
||||
command_timeout_time=${CommandTimeoutTime:-""}
|
||||
file_extensions=${RepoFileExtensions:-"js py"}
|
||||
proxy_url=${ProxyUrl:-""}
|
||||
current_branch=${QL_BRANCH:-""}
|
||||
|
||||
if [[ -n "${DefaultCronRule}" ]]; then
|
||||
default_cron="${DefaultCronRule}"
|
||||
@@ -272,14 +272,35 @@ random_range() {
|
||||
|
||||
delete_pm2() {
|
||||
cd $dir_root
|
||||
pm2 delete ecosystem.config.js
|
||||
# Try to delete PM2 processes, but don't fail if PM2 is not available
|
||||
pm2 delete ecosystem.config.js 2>/dev/null || true
|
||||
# Also try to kill any directly spawned node processes
|
||||
pkill -f "node.*static/build/app.js" 2>/dev/null || true
|
||||
}
|
||||
|
||||
reload_pm2() {
|
||||
cd $dir_root
|
||||
restore_env_vars
|
||||
pm2 flush &>/dev/null
|
||||
pm2 startOrGracefulReload ecosystem.config.js --update-env
|
||||
|
||||
# Try to start PM2, but handle failures gracefully
|
||||
if pm2 flush &>/dev/null && pm2 startOrGracefulReload ecosystem.config.js --update-env; then
|
||||
return 0
|
||||
else
|
||||
local exit_code=$?
|
||||
echo "警告: PM2 启动失败 (退出码: $exit_code),可能是由于硬件不兼容"
|
||||
echo "正在尝试直接使用 Node.js 启动服务..."
|
||||
|
||||
# Kill any existing node processes for qinglong
|
||||
pkill -f "node.*static/build/app.js" 2>/dev/null || true
|
||||
|
||||
# Start node directly in the background
|
||||
nohup node static/build/app.js > $dir_log/qinglong.log 2>&1 &
|
||||
local node_pid=$!
|
||||
|
||||
echo "已使用 Node.js 直接启动服务 (PID: $node_pid)"
|
||||
echo "注意: 使用此模式时,部分 PM2 管理功能将不可用"
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
diff_time() {
|
||||
|
||||
+19
-10
@@ -46,18 +46,22 @@ handle_log_path() {
|
||||
|
||||
time=$(date "+$mtime_format")
|
||||
log_time=$(format_log_time "$mtime_format" "$time")
|
||||
log_dir_tmp="${file_param##*/}"
|
||||
if [[ $file_param =~ "/" ]]; then
|
||||
if [[ $file_param == /* ]]; then
|
||||
log_dir_tmp_path="${file_param:1}"
|
||||
else
|
||||
log_dir_tmp_path="${file_param}"
|
||||
if [[ -z $log_name ]]; then
|
||||
log_dir_tmp="${file_param##*/}"
|
||||
if [[ $file_param =~ "/" ]]; then
|
||||
if [[ $file_param == /* ]]; then
|
||||
log_dir_tmp_path="${file_param:1}"
|
||||
else
|
||||
log_dir_tmp_path="${file_param}"
|
||||
fi
|
||||
fi
|
||||
log_dir_tmp_path="${log_dir_tmp_path%/*}"
|
||||
log_dir_tmp_path="${log_dir_tmp_path##*/}"
|
||||
[[ $log_dir_tmp_path ]] && log_dir_tmp="${log_dir_tmp_path}_${log_dir_tmp}"
|
||||
log_dir="${log_dir_tmp%.*}${suffix}"
|
||||
else
|
||||
log_dir="$log_name"
|
||||
fi
|
||||
log_dir_tmp_path="${log_dir_tmp_path%/*}"
|
||||
log_dir_tmp_path="${log_dir_tmp_path##*/}"
|
||||
[[ $log_dir_tmp_path ]] && log_dir_tmp="${log_dir_tmp_path}_${log_dir_tmp}"
|
||||
log_dir="${log_dir_tmp%.*}${suffix}"
|
||||
log_path="$log_dir/$log_time.log"
|
||||
|
||||
if [[ ${real_log_path:=} ]]; then
|
||||
@@ -73,6 +77,11 @@ handle_log_path() {
|
||||
if [[ "${real_time:=}" == "true" ]]; then
|
||||
cmd=""
|
||||
fi
|
||||
|
||||
if [[ "${log_dir:=}" == "/dev/null" ]]; then
|
||||
cmd=">> /dev/null"
|
||||
log_path="/dev/null"
|
||||
fi
|
||||
}
|
||||
|
||||
format_params() {
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
dir_shell=$QL_DIR/shell
|
||||
. $dir_shell/share.sh
|
||||
. $dir_shell/api.sh
|
||||
load_ql_envs
|
||||
. $dir_shell/env.sh
|
||||
|
||||
send_mark=$dir_shell/send_mark
|
||||
|
||||
+12
-5
@@ -7,11 +7,18 @@ export function rootContainer(container: any) {
|
||||
'en': require('./locales/en-US.json'),
|
||||
'zh': require('./locales/zh-CN.json'),
|
||||
};
|
||||
let currentLocale = intl.determineLocale({
|
||||
urlLocaleKey: 'lang',
|
||||
cookieLocaleKey: 'lang',
|
||||
localStorageLocaleKey: 'lang',
|
||||
}).slice(0, 2);
|
||||
let currentLocale: string;
|
||||
try {
|
||||
currentLocale = intl.determineLocale({
|
||||
urlLocaleKey: 'lang',
|
||||
cookieLocaleKey: 'lang',
|
||||
localStorageLocaleKey: 'lang',
|
||||
}).slice(0, 2);
|
||||
} catch (e: unknown) {
|
||||
// Handle decodeURIComponent errors from malformed cookies
|
||||
console.warn('Failed to determine locale from cookies:', e);
|
||||
currentLocale = '';
|
||||
}
|
||||
|
||||
if (!currentLocale || !Object.keys(locales).includes(currentLocale)) {
|
||||
currentLocale = 'zh';
|
||||
|
||||
+16
-11
@@ -1,6 +1,6 @@
|
||||
import intl from 'react-intl-universal';
|
||||
import React, { useRef, useState, useEffect } from 'react';
|
||||
import { Tooltip, Typography } from 'antd';
|
||||
import { Tooltip, Typography, message } from 'antd';
|
||||
import { CopyOutlined, CheckOutlined } from '@ant-design/icons';
|
||||
import { CopyToClipboard } from 'react-copy-to-clipboard';
|
||||
|
||||
@@ -10,16 +10,21 @@ const Copy = ({ text }: { text: string }) => {
|
||||
const [copied, setCopied] = useState(false);
|
||||
const copyIdRef = useRef<number>();
|
||||
|
||||
const copyText = (e?: React.MouseEvent) => {
|
||||
const handleCopy = (text: string, result: boolean) => {
|
||||
if (result) {
|
||||
setCopied(true);
|
||||
message.success(intl.get('复制成功'));
|
||||
|
||||
cleanCopyId();
|
||||
copyIdRef.current = window.setTimeout(() => {
|
||||
setCopied(false);
|
||||
}, 3000);
|
||||
}
|
||||
};
|
||||
|
||||
const handleClick = (e?: React.MouseEvent) => {
|
||||
e?.preventDefault();
|
||||
e?.stopPropagation();
|
||||
|
||||
setCopied(true);
|
||||
|
||||
cleanCopyId();
|
||||
copyIdRef.current = window.setTimeout(() => {
|
||||
setCopied(false);
|
||||
}, 3000);
|
||||
};
|
||||
|
||||
const cleanCopyId = () => {
|
||||
@@ -27,8 +32,8 @@ const Copy = ({ text }: { text: string }) => {
|
||||
};
|
||||
|
||||
return (
|
||||
<Link onClick={copyText} style={{ marginLeft: 1 }}>
|
||||
<CopyToClipboard text={text}>
|
||||
<Link onClick={handleClick} style={{ marginLeft: 4 }}>
|
||||
<CopyToClipboard text={text} onCopy={handleCopy}>
|
||||
<Tooltip
|
||||
key="copy"
|
||||
title={copied ? intl.get('复制成功') : intl.get('复制')}
|
||||
|
||||
+13
-1
@@ -521,5 +521,17 @@
|
||||
"远程仓库缓存": "Remote repository cache",
|
||||
"SSH 文件缓存": "SSH file cache",
|
||||
"清除依赖缓存": "Clean dependency cache",
|
||||
"清除成功": "Clean successful"
|
||||
"清除成功": "Clean successful",
|
||||
"日志名称": "Log Name",
|
||||
"自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成": "Custom log folder name to distinguish logs from different tasks. Leave blank to auto-generate",
|
||||
"自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成。支持绝对路径如 /dev/null": "Custom log folder name to distinguish logs from different tasks. Leave blank to auto-generate. Supports absolute paths like /dev/null",
|
||||
"自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成。支持 /dev/null 丢弃日志,其他绝对路径必须在日志目录内": "Custom log folder name to distinguish logs from different tasks. Leave blank to auto-generate. Supports /dev/null to discard logs, other absolute paths must be within log directory",
|
||||
"请输入自定义日志文件夹名称": "Please enter a custom log folder name",
|
||||
"请输入自定义日志文件夹名称或绝对路径": "Please enter a custom log folder name or absolute path",
|
||||
"请输入自定义日志文件夹名称或 /dev/null": "Please enter a custom log folder name or /dev/null",
|
||||
"日志名称只能包含字母、数字、下划线和连字符": "Log name can only contain letters, numbers, underscores and hyphens",
|
||||
"日志名称不能超过100个字符": "Log name cannot exceed 100 characters",
|
||||
"未启用": "Not enabled",
|
||||
"默认为 CPU 个数": "Default is the number of CPUs",
|
||||
"Minimum is 4": "Minimum is 4"
|
||||
}
|
||||
|
||||
+13
-1
@@ -521,5 +521,17 @@
|
||||
"远程仓库缓存": "远程仓库缓存",
|
||||
"SSH 文件缓存": "SSH 文件缓存",
|
||||
"清除依赖缓存": "清除依赖缓存",
|
||||
"清除成功": "清除成功"
|
||||
"清除成功": "清除成功",
|
||||
"日志名称": "日志名称",
|
||||
"自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成": "自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成",
|
||||
"自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成。支持绝对路径如 /dev/null": "自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成。支持绝对路径如 /dev/null",
|
||||
"自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成。支持 /dev/null 丢弃日志,其他绝对路径必须在日志目录内": "自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成。支持 /dev/null 丢弃日志,其他绝对路径必须在日志目录内",
|
||||
"请输入自定义日志文件夹名称": "请输入自定义日志文件夹名称",
|
||||
"请输入自定义日志文件夹名称或绝对路径": "请输入自定义日志文件夹名称或绝对路径",
|
||||
"请输入自定义日志文件夹名称或 /dev/null": "请输入自定义日志文件夹名称或 /dev/null",
|
||||
"日志名称只能包含字母、数字、下划线和连字符": "日志名称只能包含字母、数字、下划线和连字符",
|
||||
"日志名称不能超过100个字符": "日志名称不能超过100个字符",
|
||||
"未启用": "未启用",
|
||||
"默认为 CPU 个数": "默认为 CPU 个数",
|
||||
"最小是 4": "最小是 4"
|
||||
}
|
||||
|
||||
+32
-17
@@ -66,6 +66,7 @@ const SHOW_TAB_COUNT = 10;
|
||||
|
||||
const Crontab = () => {
|
||||
const { headerStyle, isPhone, theme } = useOutletContext<SharedContext>();
|
||||
const [allSubscriptions, setAllSubscriptions] = useState<any[]>([]);
|
||||
const columns: ColumnProps<ICrontab>[] = [
|
||||
{
|
||||
title: intl.get('名称'),
|
||||
@@ -247,8 +248,8 @@ const Crontab = () => {
|
||||
>
|
||||
{record.last_execution_time
|
||||
? dayjs(record.last_execution_time * 1000).format(
|
||||
'YYYY-MM-DD HH:mm:ss',
|
||||
)
|
||||
'YYYY-MM-DD HH:mm:ss',
|
||||
)
|
||||
: '-'}
|
||||
</span>
|
||||
);
|
||||
@@ -272,6 +273,12 @@ const Crontab = () => {
|
||||
title: intl.get('关联订阅'),
|
||||
width: 185,
|
||||
render: (text, record: any) => record?.subscription?.name || '-',
|
||||
key: 'sub_id',
|
||||
dataIndex: 'sub_id',
|
||||
filters: allSubscriptions.map((sub) => ({
|
||||
text: sub.name || sub.alias,
|
||||
value: sub.id,
|
||||
})),
|
||||
},
|
||||
{
|
||||
title: intl.get('操作'),
|
||||
@@ -361,11 +368,10 @@ const Crontab = () => {
|
||||
const getCrons = () => {
|
||||
setLoading(true);
|
||||
const { page, size, sorter, filters } = pageConf;
|
||||
let url = `${
|
||||
config.apiPrefix
|
||||
}crons?searchValue=${searchText}&page=${page}&size=${size}&filters=${JSON.stringify(
|
||||
filters,
|
||||
)}`;
|
||||
let url = `${config.apiPrefix
|
||||
}crons?searchValue=${searchText}&page=${page}&size=${size}&filters=${JSON.stringify(
|
||||
filters,
|
||||
)}`;
|
||||
if (sorter && sorter.column && sorter.order) {
|
||||
url += `&sorter=${JSON.stringify({
|
||||
field: sorter.column.key,
|
||||
@@ -523,9 +529,8 @@ const Crontab = () => {
|
||||
|
||||
const enabledOrDisabledCron = (record: any, index: number) => {
|
||||
Modal.confirm({
|
||||
title: `确认${
|
||||
record.isDisabled === 1 ? intl.get('启用') : intl.get('禁用')
|
||||
}`,
|
||||
title: `确认${record.isDisabled === 1 ? intl.get('启用') : intl.get('禁用')
|
||||
}`,
|
||||
content: (
|
||||
<>
|
||||
{intl.get('确认')}
|
||||
@@ -540,8 +545,7 @@ const Crontab = () => {
|
||||
onOk() {
|
||||
request
|
||||
.put(
|
||||
`${config.apiPrefix}crons/${
|
||||
record.isDisabled === 1 ? 'enable' : 'disable'
|
||||
`${config.apiPrefix}crons/${record.isDisabled === 1 ? 'enable' : 'disable'
|
||||
}`,
|
||||
[record.id],
|
||||
)
|
||||
@@ -565,9 +569,8 @@ const Crontab = () => {
|
||||
|
||||
const pinOrUnPinCron = (record: any, index: number) => {
|
||||
Modal.confirm({
|
||||
title: `确认${
|
||||
record.isPinned === 1 ? intl.get('取消置顶') : intl.get('置顶')
|
||||
}`,
|
||||
title: `确认${record.isPinned === 1 ? intl.get('取消置顶') : intl.get('置顶')
|
||||
}`,
|
||||
content: (
|
||||
<>
|
||||
{intl.get('确认')}
|
||||
@@ -582,8 +585,7 @@ const Crontab = () => {
|
||||
onOk() {
|
||||
request
|
||||
.put(
|
||||
`${config.apiPrefix}crons/${
|
||||
record.isPinned === 1 ? 'unpin' : 'pin'
|
||||
`${config.apiPrefix}crons/${record.isPinned === 1 ? 'unpin' : 'pin'
|
||||
}`,
|
||||
[record.id],
|
||||
)
|
||||
@@ -799,8 +801,20 @@ const Crontab = () => {
|
||||
}
|
||||
}, [viewConf, enabledCronViews]);
|
||||
|
||||
const getAllSubscriptions = () => {
|
||||
request
|
||||
.get(`${config.apiPrefix}subscriptions`)
|
||||
.then(({ code, data }) => {
|
||||
if (code === 200) {
|
||||
setAllSubscriptions(data || []);
|
||||
}
|
||||
})
|
||||
.catch(() => {});
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
getCronViews();
|
||||
getAllSubscriptions();
|
||||
}, []);
|
||||
|
||||
const viewAction = (key: string) => {
|
||||
@@ -1014,6 +1028,7 @@ const Crontab = () => {
|
||||
)}
|
||||
<Table
|
||||
columns={columns}
|
||||
sortDirections={['descend', 'ascend']}
|
||||
pagination={{
|
||||
current: pageConf.page,
|
||||
pageSize: pageConf.size,
|
||||
|
||||
@@ -55,7 +55,7 @@ const CronLogModal = ({
|
||||
const log = data as string;
|
||||
setValue(log || intl.get("暂无日志"));
|
||||
const hasNext = Boolean(
|
||||
log && !logEnded(log) && !log.includes("日志不存在"),
|
||||
log && !logEnded(log) && !log.includes("日志不存在") && !log.includes("日志设置为忽略"),
|
||||
);
|
||||
if (!hasNext && !logEnded(value) && value !== intl.get("启动中...")) {
|
||||
setTimeout(() => {
|
||||
|
||||
@@ -3,7 +3,7 @@ import config from '@/utils/config';
|
||||
import { request } from '@/utils/http';
|
||||
import { MinusCircleOutlined, PlusOutlined } from '@ant-design/icons';
|
||||
import { Button, Form, Input, Modal, Select, Space, message } from 'antd';
|
||||
import cronParse from 'cron-parser';
|
||||
import { CronExpressionParser } from 'cron-parser';
|
||||
import { useEffect, useState } from 'react';
|
||||
import intl from 'react-intl-universal';
|
||||
import { getScheduleType, scheduleTypeMap } from './const';
|
||||
@@ -91,7 +91,7 @@ const CronModal = ({
|
||||
{ required: true },
|
||||
{
|
||||
validator: (_, value) => {
|
||||
if (!value || cronParse.parseExpression(value).hasNext()) {
|
||||
if (!value || CronExpressionParser.parse(value).hasNext()) {
|
||||
return Promise.resolve();
|
||||
}
|
||||
return Promise.reject(intl.get('Cron表达式格式有误'));
|
||||
@@ -180,6 +180,35 @@ const CronModal = ({
|
||||
<Form.Item name="labels" label={intl.get('标签')}>
|
||||
<EditableTagGroup />
|
||||
</Form.Item>
|
||||
<Form.Item
|
||||
name="log_name"
|
||||
label={intl.get('日志名称')}
|
||||
tooltip={intl.get(
|
||||
'自定义日志文件夹名称,用于区分不同任务的日志,留空则自动生成。支持 /dev/null 丢弃日志,其他绝对路径必须在日志目录内',
|
||||
)}
|
||||
rules={[
|
||||
{
|
||||
validator: (_, value) => {
|
||||
if (!value) return Promise.resolve();
|
||||
if (value === '/dev/null') return Promise.resolve();
|
||||
if (value.length > 100) {
|
||||
return Promise.reject(intl.get('日志名称不能超过100个字符'));
|
||||
}
|
||||
if (!/^(?!.*(?:^|\/)\.{1,2}(?:\/|$))(?:\/)?(?:[\w.-]+\/)*[\w.-]+\/?$/.test(value)) {
|
||||
return Promise.reject(
|
||||
intl.get('日志名称只能包含字母、数字、下划线和连字符'),
|
||||
);
|
||||
}
|
||||
return Promise.resolve();
|
||||
},
|
||||
},
|
||||
]}
|
||||
>
|
||||
<Input
|
||||
placeholder={intl.get('请输入自定义日志文件夹名称或 /dev/null')}
|
||||
maxLength={200}
|
||||
/>
|
||||
</Form.Item>
|
||||
<Form.Item
|
||||
name="task_before"
|
||||
label={intl.get('执行前')}
|
||||
@@ -312,4 +341,3 @@ const CronLabelModal = ({
|
||||
};
|
||||
|
||||
export { CronLabelModal, CronModal as default };
|
||||
|
||||
|
||||
Vendored
+111
-36
@@ -1,47 +1,42 @@
|
||||
import intl from 'react-intl-universal';
|
||||
import React, {
|
||||
useCallback,
|
||||
useRef,
|
||||
useState,
|
||||
useEffect,
|
||||
useMemo,
|
||||
} from 'react';
|
||||
import useTableScrollHeight from '@/hooks/useTableScrollHeight';
|
||||
import { SharedContext } from '@/layouts';
|
||||
import config from '@/utils/config';
|
||||
import { request } from '@/utils/http';
|
||||
import { exportJson } from '@/utils/index';
|
||||
import {
|
||||
Button,
|
||||
message,
|
||||
Modal,
|
||||
Table,
|
||||
Tag,
|
||||
Space,
|
||||
Typography,
|
||||
Tooltip,
|
||||
Input,
|
||||
UploadProps,
|
||||
Upload,
|
||||
} from 'antd';
|
||||
import {
|
||||
EditOutlined,
|
||||
DeleteOutlined,
|
||||
SyncOutlined,
|
||||
CheckCircleOutlined,
|
||||
DeleteOutlined,
|
||||
EditOutlined,
|
||||
PushpinFilled,
|
||||
PushpinOutlined,
|
||||
StopOutlined,
|
||||
UploadOutlined,
|
||||
} from '@ant-design/icons';
|
||||
import config from '@/utils/config';
|
||||
import { PageContainer } from '@ant-design/pro-layout';
|
||||
import { request } from '@/utils/http';
|
||||
import EnvModal from './modal';
|
||||
import EditNameModal from './editNameModal';
|
||||
import { useOutletContext } from '@umijs/max';
|
||||
import {
|
||||
Button,
|
||||
Input,
|
||||
Modal,
|
||||
Space,
|
||||
Table,
|
||||
Tag,
|
||||
Tooltip,
|
||||
Typography,
|
||||
Upload,
|
||||
UploadProps,
|
||||
message,
|
||||
} from 'antd';
|
||||
import dayjs from 'dayjs';
|
||||
import React, { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import { DndProvider, useDrag, useDrop } from 'react-dnd';
|
||||
import { HTML5Backend } from 'react-dnd-html5-backend';
|
||||
import './index.less';
|
||||
import { exportJson } from '@/utils/index';
|
||||
import { useOutletContext } from '@umijs/max';
|
||||
import { SharedContext } from '@/layouts';
|
||||
import useTableScrollHeight from '@/hooks/useTableScrollHeight';
|
||||
import Copy from '../../components/copy';
|
||||
import intl from 'react-intl-universal';
|
||||
import { useVT } from 'virtualizedtableforantd4';
|
||||
import dayjs from 'dayjs';
|
||||
import Copy from '../../components/copy';
|
||||
import EditNameModal from './editNameModal';
|
||||
import './index.less';
|
||||
import EnvModal from './modal';
|
||||
|
||||
const { Paragraph } = Typography;
|
||||
const { Search } = Input;
|
||||
@@ -59,11 +54,15 @@ enum StatusColor {
|
||||
enum OperationName {
|
||||
'启用',
|
||||
'禁用',
|
||||
'置顶',
|
||||
'取消置顶',
|
||||
}
|
||||
|
||||
enum OperationPath {
|
||||
'enable',
|
||||
'disable',
|
||||
'pin',
|
||||
'unpin',
|
||||
}
|
||||
|
||||
const type = 'DragableBodyRow';
|
||||
@@ -181,7 +180,7 @@ const Env = () => {
|
||||
{
|
||||
title: intl.get('操作'),
|
||||
key: 'action',
|
||||
width: 120,
|
||||
width: 160,
|
||||
render: (text: string, record: any, index: number) => {
|
||||
const isPc = !isPhone;
|
||||
return (
|
||||
@@ -208,6 +207,23 @@ const Env = () => {
|
||||
)}
|
||||
</a>
|
||||
</Tooltip>
|
||||
<Tooltip
|
||||
title={
|
||||
isPc
|
||||
? record.isPinned === 1
|
||||
? intl.get('取消置顶')
|
||||
: intl.get('置顶')
|
||||
: ''
|
||||
}
|
||||
>
|
||||
<a onClick={() => pinOrUnpinEnv(record, index)}>
|
||||
{record.isPinned === 1 ? (
|
||||
<PushpinFilled />
|
||||
) : (
|
||||
<PushpinOutlined />
|
||||
)}
|
||||
</a>
|
||||
</Tooltip>
|
||||
<Tooltip title={isPc ? intl.get('删除') : ''}>
|
||||
<a onClick={() => deleteEnv(record, index)}>
|
||||
<DeleteOutlined />
|
||||
@@ -305,6 +321,51 @@ const Env = () => {
|
||||
setIsModalVisible(true);
|
||||
};
|
||||
|
||||
const pinOrUnpinEnv = (record: any, index: number) => {
|
||||
Modal.confirm({
|
||||
title: `确认${
|
||||
record.isPinned === 1 ? intl.get('取消置顶') : intl.get('置顶')
|
||||
}`,
|
||||
content: (
|
||||
<>
|
||||
{intl.get('确认')}
|
||||
{record.isPinned === 1 ? intl.get('取消置顶') : intl.get('置顶')}
|
||||
Env{' '}
|
||||
<Paragraph
|
||||
style={{ wordBreak: 'break-all', display: 'inline' }}
|
||||
ellipsis={{ rows: 6, expandable: true }}
|
||||
type="warning"
|
||||
copyable
|
||||
>
|
||||
{record.name}: {record.value}
|
||||
</Paragraph>{' '}
|
||||
{intl.get('吗')}
|
||||
</>
|
||||
),
|
||||
onOk() {
|
||||
request
|
||||
.put(
|
||||
`${config.apiPrefix}envs/${
|
||||
record.isPinned === 1 ? 'unpin' : 'pin'
|
||||
}`,
|
||||
[record.id],
|
||||
)
|
||||
.then(({ code, data }) => {
|
||||
if (code === 200) {
|
||||
message.success(
|
||||
`${
|
||||
record.isPinned === 1
|
||||
? intl.get('取消置顶')
|
||||
: intl.get('置顶')
|
||||
}${intl.get('成功')}`,
|
||||
);
|
||||
getEnvs();
|
||||
}
|
||||
});
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
const deleteEnv = (record: any, index: number) => {
|
||||
Modal.confirm({
|
||||
title: intl.get('确认删除'),
|
||||
@@ -589,6 +650,20 @@ const Env = () => {
|
||||
>
|
||||
{intl.get('批量禁用')}
|
||||
</Button>
|
||||
<Button
|
||||
type="primary"
|
||||
onClick={() => operateEnvs(2)}
|
||||
style={{ marginLeft: 8, marginBottom: 5 }}
|
||||
>
|
||||
{intl.get('批量置顶')}
|
||||
</Button>
|
||||
<Button
|
||||
type="primary"
|
||||
onClick={() => operateEnvs(3)}
|
||||
style={{ marginLeft: 8, marginRight: 8 }}
|
||||
>
|
||||
{intl.get('批量取消置顶')}
|
||||
</Button>
|
||||
<span style={{ marginLeft: 8 }}>
|
||||
{intl.get('已选择')}
|
||||
<a>{selectedRowIds?.length}</a>
|
||||
|
||||
@@ -49,6 +49,7 @@ const Setting = () => {
|
||||
reloadTheme,
|
||||
systemInfo,
|
||||
} = useOutletContext<SharedContext>();
|
||||
console.log('user',user)
|
||||
const columns = [
|
||||
{
|
||||
title: intl.get('名称'),
|
||||
|
||||
@@ -240,6 +240,7 @@ const Other = ({
|
||||
addonBefore={intl.get('每')}
|
||||
addonAfter={intl.get('天')}
|
||||
style={{ width: 180 }}
|
||||
placeholder={intl.get('未启用')}
|
||||
min={0}
|
||||
value={systemConfig?.logRemoveFrequency}
|
||||
onChange={(value) => {
|
||||
@@ -261,8 +262,9 @@ const Other = ({
|
||||
<Input.Group compact>
|
||||
<InputNumber
|
||||
style={{ width: 180 }}
|
||||
min={1}
|
||||
min={4}
|
||||
value={systemConfig?.cronConcurrency}
|
||||
placeholder={intl.get('默认为 CPU 个数')}
|
||||
onChange={(value) => {
|
||||
setSystemConfig({ ...systemConfig, cronConcurrency: value });
|
||||
}}
|
||||
|
||||
@@ -12,7 +12,7 @@ import {
|
||||
} from 'antd';
|
||||
import { request } from '@/utils/http';
|
||||
import config from '@/utils/config';
|
||||
import cron_parser from 'cron-parser';
|
||||
import { CronExpressionParser } from 'cron-parser';
|
||||
import isNil from 'lodash/isNil';
|
||||
|
||||
const { Option } = Select;
|
||||
@@ -224,8 +224,8 @@ const SubscriptionModal = ({
|
||||
type === 'raw'
|
||||
? 'file'
|
||||
: url.startsWith('http')
|
||||
? 'public-repo'
|
||||
: 'private-repo';
|
||||
? 'public-repo'
|
||||
: 'private-repo';
|
||||
|
||||
form.setFieldsValue({
|
||||
type: _type,
|
||||
@@ -381,7 +381,7 @@ const SubscriptionModal = ({
|
||||
if (
|
||||
scheduleType === 'interval' ||
|
||||
!value ||
|
||||
cron_parser.parseExpression(value).hasNext()
|
||||
CronExpressionParser.parse(value).hasNext()
|
||||
) {
|
||||
return Promise.resolve();
|
||||
} else {
|
||||
|
||||
+6
-6
@@ -1,6 +1,6 @@
|
||||
import intl from 'react-intl-universal';
|
||||
import { LANG_MAP, LOG_END_SYMBOL } from './const';
|
||||
import cron_parser from 'cron-parser';
|
||||
import { CronExpressionParser } from 'cron-parser';
|
||||
import { ICrontab } from '@/pages/crontab/type';
|
||||
|
||||
export default function browserType() {
|
||||
@@ -155,9 +155,9 @@ export default function browserType() {
|
||||
shell === 'none'
|
||||
? {}
|
||||
: {
|
||||
shell, // wechat qq uc 360 2345 sougou liebao maxthon
|
||||
shellVs,
|
||||
},
|
||||
shell, // wechat qq uc 360 2345 sougou liebao maxthon
|
||||
shellVs,
|
||||
},
|
||||
);
|
||||
|
||||
console.log(
|
||||
@@ -333,11 +333,11 @@ export function getCommandScript(
|
||||
|
||||
export function parseCrontab(schedule: string): Date | null {
|
||||
try {
|
||||
const time = cron_parser.parseExpression(schedule);
|
||||
const time = CronExpressionParser.parse(schedule);
|
||||
if (time) {
|
||||
return time.next().toDate();
|
||||
}
|
||||
} catch (error) {}
|
||||
} catch (error) { }
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user