Compare commits

..
Author SHA1 Message Date
copilot-swe-agent[bot]andGitHub cb93a1f0d3 fix: support non-root user container startup
- Guard /etc/resolv.conf write and crond behind root check in entrypoint;
  non-root containers now stay alive via 'tail -f /dev/null' instead of
  failing when crond exits with EPERM
- Set PM2_HOME to ${QL_DIR}/data/.pm2 (inside the data volume) so PM2
  does not fall back to /root/.pm2, which is inaccessible to non-root users
- Pre-create /ql/.tmp and /ql/shell/preload during image build and make
  them world-writable so non-root processes can write runtime files
- Wrap directory creation in initFile.ts with try/catch + recursive:true
  so a permission error on ~/.ssh (HOME=/root for non-root user) is logged
  as a warning instead of crashing the server init
2026-05-24 06:39:38 +00:00
d25340a857 fix: disable COOP and Origin-Agent-Cluster headers to fix blank page on HTTP
Agent-Logs-Url: https://github.com/whyour/qinglong/sessions/71ccc655-4ea0-4735-9e7c-9230000e08d5

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2026-04-25 06:57:53 +00:00
copilot-swe-agent[bot]andGitHub 173c6cb234 Initial plan 2026-04-25 06:46:52 +00:00
07bf0c705b fix: respect QlPort env var in Docker health check (#2963)
* Initial plan

* fix: use QlPort env variable in health check with fallback to default 5700

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2026-03-11 20:43:56 +08:00
whyour fd516977e3 chore: upgrade nodemailer 2026-03-07 22:35:18 +08:00
whyour c39f4ef846 chore: 更新 multer,解决 cve 漏洞 2026-03-07 21:31:24 +08:00
whyour 275d8af4e2 更新版本 v2.20.2 2026-03-01 20:35:25 +08:00
whyour 544c432f49 修复 PATH 环境变量 2026-03-01 20:35:19 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>whyour
6bec52dca1 Fix /open/user/init auth bypass allowing credential reset on initialized systems (#2941)
* Initial plan

* fix: add /open/user/init paths to init guard to prevent auth bypass

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
Co-authored-by: whyour <imwhyour@gmail.com>
2026-03-01 18:02:21 +08:00
rockymelodyandGitHub ce599d306f 青龙面板鉴权绕过漏洞已修复 (#2935)
已实施的安全加固措施
第一层防御:启用Express严格路由(第17-18行)
app.set('case sensitive routing', true);  // 路由大小写敏感
app.set('strict routing', true);           // 严格路由匹配
第二层防御:路径标准化检查中间件(第23-37行)
app.use((req, res, next) => {
  const originalPath = req.path;
  const normalizedPath = originalPath.toLowerCase();

  // 检测并拦截大小写混淆攻击
  if (originalPath !== normalizedPath &&
      (normalizedPath.startsWith('/api/') || normalizedPath.startsWith('/open/'))) {
    return res.status(400).json({
      code: 400,
      message: 'Invalid path format'
    });
  }

  next();
});
作用:主动检测并拒绝含有大小写变体的恶意请求
第三层防御:JWT中间件正则表达式修复(第59行)
// 修复前:
path: [...config.apiWhiteList, /^\/(?!api\/).*/],

// 修复后:添加大小写不敏感标志 'i'
path: [...config.apiWhiteList, /^(\/(?!api\/).*)$/i],
作用:防御正则匹配层面的绕过
第四层防御:自定义Token中间件路径标准化(第74-87行)
// 修复前:
if (!['/open/', '/api/'].some((x) => req.path.startsWith(x))) {

// 修复后:统一转小写比较
const pathLower = req.path.toLowerCase();
if (!['/open/', '/api/'].some((x) => pathLower.startsWith(x))) {
}
作用:确保Token验证逻辑对所有路径变体生效

第五层防御:初始化接口路径检查修复(第122-123行)
// 修复前:
if (!['/api/user/init', '/api/user/notification/init'].includes(req.path)) {

// 修复后:
const pathLower = req.path.toLowerCase();
if (!['/api/user/init', '/api/user/notification/init'].includes(pathLower)) {
2026-03-01 17:44:03 +08:00
whyour d53437d169 更新 2.20.1 2025-12-26 21:17:30 +08:00
whyour d526602d19 修复运行中任务停止操作 2025-12-26 01:07:08 +08:00
whyour 91b44914f6 修复环境变量排序 2025-12-26 00:41:32 +08:00
whyour 4f6c93cc1c 更新 workflow 2025-12-24 01:03:21 +08:00
whyour e326d89571 修复 apiWhiteList 路径 2025-12-23 00:58:09 +08:00
whyour 5f0dafa010 修复 cron-parser import,websocket basepath 2025-12-23 00:28:16 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
dc0b3f2eb2 Fix QlBaseUrl: use URL rewrite for base path support (#2876)
* Initial plan

* Add QlBaseUrl support to backend routes

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix whitelist check to use base-URL-aware paths

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Update websocket and frontend to support base URL

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Address code review feedback: fix JWT regex and path construction

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix path construction: use req.path directly for whitelist check

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add clarifying comments and improve code readability

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Apply code review suggestions: improve clarity and simplify logic

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Simplify baseUrl implementation using URL rewrite

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-12-22 23:44:29 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
3db716763d Fix cron-parser v5 bundling incompatibility causing validation failures (#2877)
* Initial plan

* Fix: Use default import for cron-parser to ensure browser compatibility

Changed from named export `{ CronExpressionParser }` to default export `cronParser` and access `CronExpressionParser` through it. This ensures compatibility with webpack/UmiJS bundling for browser environments while maintaining backend functionality.

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-12-22 23:43:54 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
fae226745e Add missing larkSecret field to gRPC NotificationInfo proto (#2880)
* Initial plan

* Add larkSecret field to NotificationInfo proto definition

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-12-22 23:38:42 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
9330650163 Fix TG_PROXY_AUTH concatenation in notify.js - add missing @ separator (#2882)
* Initial plan

* Fix TG_PROXY_AUTH handling in notify.js to match notify.py logic

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Apply prettier formatting to notify.js

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-12-22 23:05:06 +08:00
CopilotGitHubwhyourwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
073de76a4a Fix validation error when saving scripts in debug window (v2.20.0 regression) (#2862)
* 更新版本 2.20.0

* Initial plan

* Fix validation error when saving scripts by allowing unknown fields in POST /scripts

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Revert version.yaml to 2.19.2 - should not include version bump in bug fix PR

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: whyour <imwhyour@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-12-22 22:43:48 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
c61d1aa828 Fix enum value 0 causing type filter to fail for NodeJS dependencies (#2869)
* Initial plan

* Fix: Prevent Python3 dependencies from appearing in NodeJs tab

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-12-15 18:21:14 +08:00
whyour 33fa3aca99 更新版本 2.20.0 2025-12-11 01:53:17 +08:00
whyour c772fc9527 修复脚本调试保存文件错误 2025-12-11 01:52:47 +08:00
whyour c5d2aa3aba 更新 pipeline 2025-12-10 00:34:35 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
02a05f06bd Add signature verification support for Feishu bot notifications (#2856)
* Initial plan

* Add signature verification support for Feishu bot notifications

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add clarifying comments about Feishu signature algorithm

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add i18n translations for larkSecret configuration field

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-27 01:10:04 +08:00
whyour 3b0f55caf4 修复任务实例默认值 2025-11-23 12:45:02 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
6a3dd4f83c Fix null log_name issue by omitting it from shell command when not set (#2849)
* Initial plan

* Fix null log_name handling in runSingle method

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Update cron.log_name before makeCommand to avoid passing null to shell

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix: Only pass log_name to shell when it has a value

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix uniqPath calculation in runSingle for null log_name

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Improve comment clarity in makeCommand

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Refactor: Move no_tee and ID to initial commandVariable declaration

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Refactor: Simplify uniqPath ternary expression

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-22 12:06:01 +08:00
whyour 177cd3de81 更新 docker 日志 2025-11-22 01:05:28 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
d473c3ae88 Fix SSH global private key matching before subscription-specific keys (#2845)
* Initial plan

* Fix SSH global private key loading order by using zzz_ prefix

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Use tilde (~) prefix for global SSH config to ensure it loads last

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-21 01:53:58 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
ee2fbe5335 Add global SSH key configuration in system settings (#2840)
* Initial plan

* Add backend support for global SSH keys

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add frontend UI for global SSH keys management

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add SshKeyModel to database initialization

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add SSH config generation for global SSH keys

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add internationalization support for SSH key management UI

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Simplify to single global SSH key in system settings

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-20 10:09:01 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
48abf44ceb feat: Support multiple concurrent login sessions per platform (#2816)
* Initial plan

* Implement multi-device login support - allow multiple concurrent sessions

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Address code review feedback - extract constants and utility functions

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add validation and logging improvements based on code review

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Revert unnecessary file changes - keep only multi-device login feature files

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-19 00:18:29 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
03c7031a3c Fix task duplication: add single/multi-instance support with UI configuration and stop all running instances (#2837)
* Initial plan

* Stop running tasks before starting new scheduled instance

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add multi-instance support and fix stop to kill all running instances

- Add allow_multiple_instances field to Crontab model (default: 0 for single instance)
- Add validation for new field in commonCronSchema
- Add getAllPids and killAllTasks utility functions
- Update stop method to kill ALL running instances of a task
- Update runCron to respect allow_multiple_instances config
- Backward compatible: defaults to single instance mode

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add UI support for allow_multiple_instances configuration

- Add allow_multiple_instances field to ICrontab interface
- Add instance mode selector in task creation/edit modal
- Add translations for instance mode in Chinese and English
- Default to single instance mode for backward compatibility

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add allow_multiple_instances column migration and optimize db.ts

- Add allow_multiple_instances column to Crontabs table migration
- Refactor migration code to use data-driven approach
- Replace 11 individual try-catch blocks with single loop
- Improve code maintainability and readability

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-19 00:10:27 +08:00
whyour 0e5de4a824 更新启动日志 2025-11-16 21:31:52 +08:00
whyour af96bd98ac 修复系统提示 2025-11-16 21:26:14 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
08ef509e27 Optimize log file writes using stream pooling (#2835)
* Initial plan

* Implement LogStreamManager for optimized log writing

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix error handler in LogStreamManager to avoid race conditions

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-16 21:11:10 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
fbeb4f4a6c Fix PM2 startup failures on ARM routers with Node.js incompatibility (#2828)
* Initial plan

* Add fallback mechanism for PM2 startup failures on incompatible hardware

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-16 20:29:32 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
eb09a417a1 Fix DNS resolution failures in Alpine containers by setting ndots:0 (#2827)
* Initial plan

* Fix DNS resolution issue in Alpine containers by adding ndots:0 option

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-16 20:28:12 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
e84ddb6cfc Fix environment variable copy reliability by validating clipboard operation (#2833)
* Initial plan

* Fix env variable copy issue by using onCopy callback

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-16 18:23:45 +08:00
whyour d01ec3b310 修复 parse cron 2025-11-15 19:09:07 +08:00
whyour 45fac29804 移除用户判断 2025-11-15 17:02:00 +08:00
whyour 72d4c8bfd4 修复 homedir 权限 2025-11-15 02:27:27 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>whyour
8f23c61fee Add non-root Docker user support with automatic command setup (#2830)
* Initial plan

* Add non-root user guide and improve error messages

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add PM2_HOME configuration to fix non-root user permission errors

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix PM2 socket error by using /tmp for PM2_HOME instead of mounted volume

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Document symlink permission limitation for non-root users

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Clarify that scheduled tasks auto-adapt to use full paths when symlinks unavailable

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add error handling for symlink creation to prevent worker crashes

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add practical solutions for non-root users to use ql and task commands

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Modify linkCommand to create symlinks in ~/bin for non-root users

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* 修复链接 task 命令

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
Co-authored-by: whyour <imwhyour@gmail.com>
2025-11-15 01:46:24 +08:00
whyour e41eed81ae 移除说明文档 2025-11-14 23:23:01 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
58f5fe763f Add cron task management to QLAPI (#2826)
* Initial plan

* Add cron management capabilities to QLAPI

- Added GetCrons, GetCronById, EnableCrons, DisableCrons, RunCrons to proto
- Implemented gRPC handlers for new cron operations
- Updated client.js to expose new methods to scripts
- Updated sample to demonstrate new API usage

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add documentation for QLAPI cron management features

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Specify pnpm version 8.3.1 in package.json

Add packageManager field to enforce pnpm@8.3.1 as requested

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Revert pnpm-lock.yaml to original state

The lockfile was inadvertently updated during dependency installation
for proto generation. Reverting to maintain consistency with pnpm@8.3.1.

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Regenerate proto files with correct tool versions

Regenerated using protoc v3.17.3 and protoc-gen-ts_proto v2.6.1
to match the original project configuration.

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-14 23:20:56 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
214241797d Fix QlPort and QlGrpcPort environment variables ignored in host network mode and pm2 reload (#2825)
* Initial plan

* Fix host mode port configuration by using QlPort environment variable

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix GRPC_PORT conflict in host network mode

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Ensure BACK_PORT and GRPC_PORT survive pm2 reload with --update-env

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Move env.sh sourcing after fix_config to preserve more environment variables

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Refactor: Extract export_ql_envs function and move env.sh sourcing earlier

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Extract load_ql_envs function and reorder initialization in docker-entrypoint.sh and update.sh

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-14 22:56:39 +08:00
whyour aedd48c9c4 修改错误日志 2025-11-14 22:23:30 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
90ddf0fb57 Fix race condition preventing scheduled tasks from executing in clustered mode (#2819)
* Initial plan

* Fix race condition causing scheduled tasks not to run

Added synchronization to ensure gRPC worker is ready before HTTP worker starts. This prevents the race condition where autosave_crontab() tries to register cron jobs before the gRPC server is ready to accept them.

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add timeout protection for gRPC worker initialization

Added 30-second timeout to prevent system hang if gRPC worker fails to start. This provides better error handling and prevents indefinite waiting.

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Refactor worker ready logic and improve restart handling

- Extracted waitForWorkerReady() method for better code reusability
- Improved worker restart logic to wait for gRPC worker readiness
- This addresses code review feedback for better maintainability

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Improve logging accuracy for worker restarts

- Fixed log messages to accurately reflect worker state
- Added proper logging after gRPC worker is confirmed ready
- Improved HTTP worker restart logging with PID
- Addresses code review feedback for better clarity

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Revert unnecessary pnpm-lock.yaml changes

The pnpm-lock.yaml was unintentionally updated when installing dependencies for testing. No package dependencies were actually changed - only existing code was modified in back/app.ts. Reverting to original state.

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Re-register cron jobs when gRPC worker restarts

When the gRPC worker restarts, the scheduled tasks need to be re-added to the new gRPC server instance. This fix:

1. Tracks the HTTP worker reference in the master process
2. Sends a 'reregister-crons' message to the HTTP worker after gRPC restarts
3. HTTP worker calls autosave_crontab() to re-register all cron jobs with the new gRPC server

This ensures scheduled tasks continue to work after a gRPC worker restart.

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-12 00:59:22 +08:00
whyour c9fc9b4b45 定时任务支持订阅筛选 2025-11-11 01:24:28 +08:00
whyour 8fdc69421c 修改定时任务支持的排序顺序 2025-11-11 01:02:36 +08:00
whyour 1deb264913 升级 cron-parser 2025-11-11 00:37:03 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
1f2fd8ac02 Fix URIError from malformed cookies causing white screen on load (#2811)
* Initial plan

* Fix decodeURIComponent error in cookie parsing by adding try-catch

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add type annotation and logging to catch block per code review

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix TypeScript errors in deps.ts - remove unused path parameter

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Revert pnpm-lock.yaml to avoid unnecessary lockfile version upgrade

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-09 21:53:06 +08:00
whyour 06aa07329f 修复日志目录逻辑 2025-11-09 21:42:45 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>whyour
4cb9f57479 环境变量支持置顶 (#2822)
* Initial plan

* Add pin to top feature for environment variables

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Format code with prettier

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add database migration for isPinned column in Envs table

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Use snake_case naming (is_pinned) for database column

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
Co-authored-by: whyour <imwhyour@gmail.com>
2025-11-09 19:43:33 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
c369514741 定时任务支持自定义日志文件或者 /dev/null (#2823)
* Initial plan

* Add log_name field to enable custom log folder naming

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add database migration for log_name column

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add security validation to prevent path traversal attacks

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Apply prettier formatting to modified files

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Support absolute paths like /dev/null for log redirection

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Restrict absolute paths to log directory except /dev/null

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-09 19:32:40 +08:00
whyour 0e28e1b6c4 修复 pm2 日志目录 2025-11-09 17:52:38 +08:00
whyour 73f8f3c5fa 增加 agent
Create a new agent configuration file for QL agent.
2025-11-08 00:05:56 +08:00
whyour 494ccb3c08 更新 workflow 2025-11-03 00:09:36 +08:00
whyour 399728b433 修复 jwt 认证 2025-11-02 22:28:58 +08:00
whyour 52a1de5063 更新 demo 地址 2025-11-02 21:54:19 +08:00
whyour 18f27a9a69 移除 nginx 2025-11-02 19:29:59 +08:00
whyour 07951964a1 修复模块注入 2025-10-26 22:32:03 +08:00
涛之雨andGitHub a1f888af59 Add validation to dependencies GET endpoint and update service logic (#2778)
* Add validation to dependencies GET endpoint and update service logic

* fix https://github.com/whyour/qinglong/pull/2778/files/6063bc3a67fb329de9b90f7c93524b862bd9eb93#r2266494581

* remove default condition type

* fix query mistakes
2025-10-11 23:23:13 +08:00
涛之雨andGitHub f7472b6e74 Add input validation to script API routes (#2777)
* Add input validation to script API routes

* 优化脚本 API 路由的错误处理逻辑

* Fix optional path compatibility checks

* remove file
2025-10-11 23:20:26 +08:00
whyour a7baeba755 修复 task 命令可能软链失败 2025-10-11 23:15:54 +08:00
涛之雨andGitHub e4f733320d Enable debug backend (#2776) 2025-08-19 11:13:33 +08:00
whyour 55c92dc320 发布版本 v2.19.2 2025-07-12 20:29:49 +08:00
whyour 50769c43dd 修复 command-run 接口日志绑定 2025-07-12 20:29:44 +08:00
whyour 0587644a6b command-run 增加返回 QL-Task-Log 日志路径 2025-06-28 01:05:05 +08:00
whyour 87b934aafe QLAPI.systemNotify 支持自定义通知类型和参数 2025-06-24 02:00:51 +08:00
whyour 7a92e7c6ab 修复取消安装依赖 2025-06-22 21:47:39 +08:00
whyour 1d8403c0ec 修复环境变量过大解析 2025-06-22 21:10:55 +08:00
whyour ef9e38f167 备份数据支持选择模块,支持清除依赖缓存 2025-06-22 14:25:19 +08:00
whyour c9bd053fbd 修改服务启动方式 2025-06-11 00:42:29 +08:00
憶夣andGitHub 57939391b9 ntfy 增加可选的认证与用户动作 (#2741)
* feat:ntfy增加可选的认证

* feat:ntfy增加可选的用户动作

* fix:ntfy动作包含中文报错
2025-06-07 00:26:27 +08:00
whyour 394e96bbf8 修复 health 接口报错 2025-06-07 00:25:47 +08:00
whyour 47c194c1f4 更新版本 v2.19.1 2025-05-24 15:03:08 +08:00
whyour 7d65d96ebd 修复 demo 环境提示 2025-05-24 14:56:49 +08:00
whyour 224000b63b 修复依赖是否安装检查逻辑 2025-05-23 23:45:43 +08:00
whyour 1c18668bad 修复文件下载参数 2025-05-22 00:09:19 +08:00
whyour f94582b68d 修复查询 python 依赖存在逻辑 2025-05-21 01:25:24 +08:00
whyour eb1c00984c 修复任务视图状态包含筛选 2025-05-20 23:40:18 +08:00
whyour 1a185f5682 修复创建脚本可能失败 2025-05-20 01:00:08 +08:00
whyour b6ea8565ec 更新版本 v2.19.0 2025-05-18 22:02:57 +08:00
whyour 2e94d58758 修改启动命令环境变量 2025-05-18 22:02:55 +08:00
whyour 472a3088df 修复启动逻辑 2025-05-17 17:25:50 +08:00
whyour ec3d61a713 修复 /:file 获取日志接口 2025-05-16 00:57:27 +08:00
whyour 95459c33ed 修改 import undici 2025-05-15 01:19:24 +08:00
whyour 5459b72f63 更新 pnpm-lock 2025-05-15 01:15:55 +08:00
whyour 05db2b1df8 修复重置用户名失败 2025-05-15 01:12:01 +08:00
whyour c3072e7712 got 替换为 uudici 2025-05-15 01:01:39 +08:00
whyour 3fafe4d24d 修复依赖强制删除未移除队列 2025-05-13 02:00:37 +08:00
whyour 32bccb3f3e 修复无法识别 python 依赖安装的命令 2025-05-13 00:22:15 +08:00
whyour ac04478d1d 修复重用户名 2025-05-11 18:59:15 +08:00
whyour 8a18baa921 修复登录通知失败造成服务重启 2025-05-11 14:36:16 +08:00
whyour 9a399f8de8 更新 workflow 2025-05-09 23:36:14 +08:00
whyour da639a1f8f 移除部分历史兼容逻辑 2025-05-09 15:37:05 +08:00
whyour 425e49675a 修复删除日志 2025-05-09 00:20:53 +08:00
whyour 8174762c18 demo 环境不自动运行任务 2025-05-08 01:43:14 +08:00
whyour 710a107e73 邮箱通知支持多个收件人 2025-05-08 01:27:25 +08:00
whyour d871585eee 修改服务启动逻辑 2025-05-07 09:30:00 +08:00
whyour 729b405b0f 修复 bootAfter 目录 2025-04-26 23:26:11 +08:00
whyour 71a7c1b9d3 boot 任务改为在依赖安装完成后执行 2025-04-25 23:52:33 +08:00
whyour 40a831f3a2 修复脚本管理查询逻辑 2025-04-25 01:40:37 +08:00
whyour 1befa1bb8c 缓存 node 和 python 依赖 2025-04-23 02:10:39 +08:00
whyour 8009634b44 修复脚本管理增加文件夹 2025-04-19 16:34:28 +08:00
whyour 03a23f6f31 更新 workflow 运行环境 2025-04-19 14:36:21 +08:00
whyour 124e01e93a 修复 QLAPI 修复环境变量 remarks,esm 依赖查不到 2025-04-19 01:38:43 +08:00
whyour 2370923a75 更新版本 v2.18.3 2025-03-19 23:47:53 +08:00
whyour 256e4e7a83 修复任务打开脚本链接 2025-03-19 23:47:51 +08:00
whyour c0ec063333 修复删除脚本参数 2025-03-18 23:17:03 +08:00
whyour 576408de01 脚本管理增加强制打开文件操作 2025-03-16 17:57:13 +08:00
whyour cf94ecfb11 脚本管理和日志管理支持下载 2025-03-13 00:22:24 +08:00
whyour 118c92d9e5 更新版本 v2.18.2 2025-02-28 00:56:43 +08:00
whyour 280085668e 修复未初始化时区设置 2025-02-28 00:56:40 +08:00
whyour bae4073a64 增加重置密码命令 2025-02-27 23:57:26 +08:00
whyour af3e358a6a 系统设置增加时区设置 2025-02-27 00:45:21 +08:00
whyour 64fcbff715 脚本管理增加可预览检查 2025-02-26 01:17:11 +08:00
whyour f9f78b4e05 修改系统内置通知模块名称,避免重复 2025-02-25 00:32:13 +08:00
whyour fa83761d27 修改定时规则类型 2025-02-21 01:35:08 +08:00
whyour 8173075b67 定时任务支持 @once 和 @boot 任务 2025-02-20 02:18:59 +08:00
whyour 496918131f 修复群晖通知参数,任务视图不属于筛选 2025-02-16 12:15:45 +08:00
whyour 5d64aae452 修复 mirror action 2025-02-07 01:56:23 +08:00
whyour 952ea2859e 修复最后运行时长多语言显示 2025-01-31 23:02:25 +08:00
whyour 25e8e8198d 修改 QLAPI 调用校验 2025-01-30 00:55:48 +08:00
whyour 3ab5b0d86b 修复登录失败没有提示 2025-01-29 23:50:46 +08:00
whyour 6df651aa63 更新 ts-proto 版本 2025-01-29 23:29:45 +08:00
whyour 924dfd6a6f 更新版本 v2.18.1 2025-01-22 01:17:15 +08:00
whyour 07f43538df 修复写入文件 mode 2025-01-22 01:17:13 +08:00
whyour 4fa5fa2014 修改 lock 文件名称规则 2025-01-16 00:35:47 +08:00
whyour f9b9543a6d 修复 JavaScript telegram 通知 2025-01-15 23:23:54 +08:00
whyour af97543918 修改错误提示 2025-01-14 23:20:53 +08:00
whyour a508522872 增加安全报告提示 2025-01-14 01:05:48 +08:00
whyour f1ca2134b7 移除 nedb 和 sentry 2025-01-14 00:24:25 +08:00
whyour a9755655b2 更新 readme 2025-01-12 18:32:48 +08:00
whyour 6a76e82f26 修改 QLAPI python 示例 2025-01-12 18:11:42 +08:00
whyour 6775f5d123 修复 QLAPI 系统通知响应 2025-01-12 17:02:21 +08:00
whyour ad6e08525c 修改 QLAPI 系统通知 2025-01-12 15:39:28 +08:00
whyour 51ef4e7476 修改任务状态更新失败提示,重复运行提示 2025-01-12 00:19:14 +08:00
whyour e5b35273f9 修复更新环境变量 2025-01-11 17:14:30 +08:00
whyour 647ed3b66c QLAPI 支持操作环境变量和系统通知 2025-01-11 01:59:46 +08:00
whyour 4667af4ebe 更新版本 v2.18.0 2025-01-05 22:03:45 +08:00
whyour 3f775a0e6c 修复创建系统应用 2025-01-05 22:03:41 +08:00
whyour fa79de3f05 修复 status 类型 2025-01-05 12:23:13 +08:00
whyour ffa8b25a66 优化初始化文件操作 2025-01-05 00:28:08 +08:00
whyour 05f8bbd26e 写入文件增加文件锁,避免竞争条件引起文件内容异常 2025-01-04 01:22:29 +08:00
whyour 7d43b14f81 修复重置登录错误次数和 tfa 2025-01-02 23:50:20 +08:00
whyour cecc5aeb15 修复初始化 SystemConfig 数据 2025-01-01 21:33:43 +08:00
whyour 43d6ac2071 增加手动停止标识 2024-12-31 00:17:30 +08:00
whyour 678e3e2dc6 修改认证信息存储方式,避免认证信息异常 2024-12-30 14:23:04 +08:00
whyour 75f91e1473 修改通知文件未设置变量提示 2024-12-22 22:15:09 +08:00
whyour bdc45c538c 更新版本 v2.17.13 2024-12-21 22:48:58 +08:00
FanchangWangandGitHub 955c7377d7 新增 wxPusher 推送通道。wxPusher 官方文档: https://wxpusher.zjiecode.com/docs/ (#2594)
* 增加 wxPusher 推送方式

* 补全 wxPusher 推送方式代码

* fix js 方法多进行了一次 JSON 格式化
2024-12-13 14:13:34 +08:00
c71abd8c86 完善pushplus通知方式的代码逻辑 (#2454)
* 1. pushplus通知方式返回结果优化
2. pushplus通知方式适配更多参数

* 细节bug修改

* 删除没用的空格

---------

Co-authored-by: 陈思远 <chensiyuan@yiban.com.cn>
Co-authored-by: 陈大人 <chensiyuan@pushplus.plus>
2024-12-13 14:12:16 +08:00
whyour ab27a4c908 更新 node 依赖 2024-12-12 00:26:11 +08:00
whyour 026640a757 修复任务列表下次运行时间展示 2024-12-03 22:35:34 +08:00
whyour a1501705c1 更新 readme 2024-11-18 23:19:15 +08:00
whyour b321530dcf 修复本地部署设置 dataPath 后,备份目录可能错误 2024-11-18 22:59:07 +08:00
xiaobingtechandGitHub 02c6ad8004 mobile增加识别鸿蒙系统 (#2571)
由于鸿蒙系统的User-Agent如下
Mozilla/5.0 (Phone; OpenHarmony5.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 ArkWeb/4.1.6.1 Mobile
导致目前登录显示desktop
2024-11-18 22:40:14 +08:00
Hugh GaoandGitHub fff572869e chore: Update docker-compose.yml (#2553) 2024-11-08 22:52:19 +08:00
whyour b733937691 更新版本 v2.17.12 2024-10-26 14:59:36 +08:00
whyour 0d9eba4b6e 定时任务外漏标签改为 10 个 2024-10-26 14:49:02 +08:00
whyour 20c6a1e8bf 移除任务执行前后的脚本参数 2024-10-26 14:12:58 +08:00
qiaoyun680GitHubqiaoyun680 <qiaoyun680>
56bc2f0b1d 改进ntfy通知,将中文标题进行编码后发送 (#2541)
* 1.改进ntfy通知,将中文标题进行编码后发送,可以直接展示中文标题;2.改进notify.py和notify.js中ntfy的推送;
---------

Co-authored-by: qiaoyun680 <qiaoyun680>
2024-10-23 16:38:51 +08:00
EasyandGitHub ecc55883f0 优化Server酱Key相关的逻辑 (#2538)
* 优化sendkey的兼容

* 对title和desp encode
2024-10-22 21:49:07 +08:00
qiaoyun680GitHubqiaoyun680 <qiaoyun680>
185fd2ff91 增加 ntfy 通知 (#2537)
Co-authored-by: qiaoyun680 <qiaoyun680>
2024-10-22 21:47:38 +08:00
whyour 3822c37fa0 修复登录日志保存 2024-10-19 00:18:24 +08:00
whyour 4244502949 增加 log 图标样式 2024-10-13 21:24:17 +08:00
EasyandGitHub 71dd82f74e 适配Server酱APP分支(Server酱³),移除已废弃的旧版api入口 (#2524) 2024-10-10 22:09:00 +08:00
whyour 6ea8d361fd 修复 mac 时间格式化 2024-09-29 22:47:57 +08:00
whyour 3f71f9acdb 修复任务命令带有 -m 参数时,日志目录生成异常 2024-09-29 22:27:52 +08:00
whyour 418695c4aa 定时任务支持复制 2024-09-24 23:33:04 +08:00
whyour 4c5223ba11 更新版本 v2.17.11 2024-09-16 21:35:38 +08:00
whyour 2f70e10be7 修复 shell 检查 nounset 变量 2024-09-16 21:35:36 +08:00
whyour 66706ce150 修复任务视图过多无法选到视图管理 2024-09-10 22:50:33 +08:00
whyour 3d719baa54 修复任务执行初始化环境变量 PYTHONPATH 和 NODE_OPTIONS 被清空 2024-09-10 22:41:39 +08:00
whyour 926ad067c1 修复通知文件换行符解析错误 2024-09-09 23:19:58 +08:00
whyour ff98c3a499 修复任务跳转脚本未显示文件大小,订阅删除未删除 repo 目录 2024-09-08 21:08:15 +08:00
whyour 69fcf45231 修复任务执行前命令错误提示 2024-09-08 17:58:37 +08:00
whyour c3704502bf 更新 readme 2024-09-05 22:35:14 +08:00
whyour a0613d0f39 修复文件越权访问 2024-09-04 23:25:48 +08:00
whyour 8c0f46420e 更新 readme 2024-09-04 22:27:03 +08:00
whyour ea4b474b83 更新版本 v2.17.10 2024-09-01 23:42:38 +08:00
whyour 53ef0fe296 修复订阅任务参数 2024-09-01 23:42:36 +08:00
whyour c573186e34 修复启动无法创建订阅任务 2024-09-01 22:47:29 +08:00
whyour c229eca315 修复依赖管理和脚本管理样式 2024-08-31 23:46:29 +08:00
whyour a41dd74048 修复 task_before/task_after 中换行后 crontab 不识别 2024-08-31 20:46:00 +08:00
whyour 459f465f3b 修复 shell 未定义变量错误 2024-08-31 20:23:37 +08:00
whyour b508e97dc1 修复依赖列表样式 2024-08-31 14:59:54 +08:00
whyour 7414a9d33d 执行任务增加结束日志 2024-08-31 14:43:44 +08:00
whyour a48d100b2d 修复表格样式 2024-08-30 00:30:31 +08:00
whyour a3044f9d29 修复 node 内置脚本运行错误 2024-08-28 00:28:25 +08:00
whyour 19cfc9e351 修复任务重复运行提示 2024-08-26 23:05:53 +08:00
whyour 6c61ac5106 修复 dockerfile 2024-08-26 00:56:55 +08:00
whyour 65f7483688 修复任务频繁运行通知 2024-08-25 16:28:32 +08:00
whyour 8b042d90f3 修复删除日志命令 2024-08-24 22:43:09 +08:00
whyour 4e5ad6d5f3 定时服务区分系统、订阅、脚本任务 2024-08-23 23:06:50 +08:00
whyour 8b8eae211b 增加任务重复运行提醒 2024-08-23 09:37:26 +08:00
whyour f4cb3eacf8 系统日志增加时间筛选和清空 2024-08-22 00:47:24 +08:00
whyour f6021c8157 增加内置 Python requests 模块 2024-08-19 23:06:16 +08:00
whyour 73601ca853 修复任务执行前命令字符转义 2024-08-19 22:27:09 +08:00
whyour 8218d4ba94 修复 Dockerfile 2024-08-18 17:26:33 +08:00
whyour d47f835531 更新 docker action 2024-08-18 17:03:49 +08:00
whyour 230a8f61b8 修复 smtp python 参数说明 2024-08-18 16:31:39 +08:00
whyour af5de8372c 修复 shell 变量初始化检查,更新 sentry 版本 2024-08-18 14:19:45 +08:00
whyour a743aa8115 更新版本 v2.17.9 2024-07-28 18:40:10 +08:00
whyour fe4516df00 修复执行 task_before 环境变量加载顺序 2024-07-28 18:40:02 +08:00
whyour be2da98674 修复任务中 task_before 参数双引号解析 2024-07-26 11:30:03 +08:00
whyour 0da8155bac 修复 desi 和 conc 命令参数解析 2024-07-25 20:53:11 +08:00
whyour eddd258614 修复 node 前置逻辑 2024-07-24 23:30:18 +08:00
whyour 2ad6437672 更新 config.sample 2024-07-22 23:37:51 +08:00
whyour 162bd9d8cc 修复 config 引入顺序 2024-07-22 23:29:08 +08:00
whyour e694570f1a 增加 task_before.js 和 task_before.py 文件,在执行任务前执行,避免环境变量过大报错 2024-07-22 01:07:00 +08:00
whyour eb5cc3943d Javascript 和 Python 增加内置函数 QLAPI.notify 2024-07-21 01:15:16 +08:00
whyour 1b39d3ab48 JavaScript 和 Python 执行脚本前执行 task_before 2024-07-20 17:28:48 +08:00
whyour f47a788184 更新版本 v2.17.8 2024-07-13 23:54:06 +08:00
whyour 738fe04ce4 修复 js py 变量分割 2024-07-13 23:53:59 +08:00
whyour a2812d9c03 移除 ts-node dependence 2024-07-13 00:49:23 +08:00
whyour 5c53ae4993 修改仓库默认拉取文件后缀,支持运行 cjs pyc 2024-07-12 01:40:37 +08:00
whyour e191aca41f 修改环境变量加载逻辑 2024-07-10 23:29:44 +08:00
whyour 5afac3a3ac 更新版本 v2.17.7 2024-07-02 00:44:56 +08:00
whyour 71ba1534f2 增加自定写入 js 和 py 类型环境变量 2024-07-02 00:44:48 +08:00
whyour 791cf657b5 修复更新 linux 镜像源错误 2024-06-30 15:09:17 +08:00
whyour c1ef021009 更新 readme 2024-06-30 00:55:10 +08:00
whyour 90fe63211d 修复修改任务状态可能报错 2024-06-28 20:41:05 +08:00
whyour b60cda66bb 更新版本 v2.17.6 2024-06-25 22:25:43 +08:00
whyour 7efe81df9e 修复 data 目录判断逻辑 2024-06-25 22:25:36 +08:00
whyour 0d492e94f4 修复未设置通知时通知报错 2024-06-23 22:22:36 +08:00
whyour a45efbd69b 修复定时任务查询不存在的订阅报错 2024-06-17 23:04:47 +08:00
whyour 035f0eb9e3 修复通知文件一言设置 2024-06-16 00:28:45 +08:00
whyour 7d0cae7839 修复单文件订阅代理无效 2024-06-13 22:53:49 +08:00
whyour 46e71d8213 修复系统设置数据未初始化 2024-06-12 13:53:45 +08:00
whyour 372afb92c6 更新 readme 2024-06-09 21:33:30 +08:00
whyour 69d9307be9 bark 推送改为 post 请求 2024-06-07 10:54:58 +08:00
whyour c3908e956f 修复下载单文件订阅 2024-06-06 22:40:51 +08:00
whyour 1e749f3731 更新版本 v2.17.5 2024-06-02 17:53:40 +08:00
whyour a68f4b0cde 后端服务指定 ipv4 端口启动 2024-06-02 17:53:37 +08:00
whyour 07541569c5 修复定时任务 real_time 参数,升级 protobufjs 2024-06-02 11:08:35 +08:00
whyour 70f2bef5d2 修改 ql 和 task 命令生成逻辑 2024-05-29 22:09:01 +08:00
whyour 805a716033 修复任务详情查看脚本错误 2024-05-25 18:22:01 +08:00
e77d4c4337 增加微加机器人消息通道 (#2373)
Co-authored-by: chensiyuan <chensiyuan@pushplus.plus>
2024-05-25 00:33:31 +08:00
whyour b309ac7613 修复有可能手动运行任务无日志 2024-05-25 00:28:55 +08:00
雨思andGitHub 8dd379b6b9 PushMe 通知支持自建服务 (#2358) 2024-05-15 22:38:25 +08:00
whyour 0b52e05af8 bark 推送增加 archive 参数 2024-05-05 20:10:57 +08:00
whyour e9852a0002 修改示例脚本 2024-05-05 19:47:54 +08:00
whyour 20be84b352 重构 JavaScript 脚本通知文件 2024-05-05 18:58:15 +08:00
whyour 4ab24ea1b9 更新版本 v2.17.4 2024-04-26 21:04:10 +08:00
whyour 417f91207f 更新 tsx 版本 2024-04-26 20:58:48 +08:00
whyour f723631647 修复脚本管理上传文件文件名乱码 2024-04-22 22:39:27 +08:00
whyour 6657ff0560 修改删除日志脚本提示 2024-04-21 22:35:20 +08:00
whyour 0593dae41d 增加示例脚本 2024-04-21 22:27:51 +08:00
whyour 445dee00f7 修复脚本和日志列表获取软链接循环失败 2024-04-21 21:59:02 +08:00
whyour 6ca28190b0 修复 text/plain body 格式化 2024-04-16 14:06:32 +08:00
whyour 510534ee0c 修复 text/plain 通知消息替换 2024-04-16 10:48:33 +08:00
whyour 2e7f3a1578 修复删除日志提示 2024-04-11 21:11:21 +08:00
Cp0204andGitHub 591a17e7ee 改进自定义 send() 方法,修正自定义通知 body 解析笔误 (#2335) 2024-04-11 21:07:49 +08:00
星野SKYandGitHub bd7c361300 更新飞书 bot 验证过程,兼容新状态码 (#2302) 2024-04-11 19:13:39 +08:00
409b3281cc 更新飞书 bot 验证过程,兼容新状态码 (#2300)
Co-authored-by: Yu Zhao <0+cmal@users.noreply.github.com>
2024-04-11 19:13:23 +08:00
whyour ac6de4911a 更新版本 v2.17.3 2024-03-29 22:45:02 +08:00
whyour 1628e05ece 复制仓库任务时,保留任务原始目录层级 2024-03-28 23:42:05 +08:00
whyour 7b7e03b503 修改 git clone 参数 2024-03-28 23:00:49 +08:00
whyour 8d14c9dae1 修复配置文件默认参数 2024-03-25 23:41:39 +08:00
whyour 4d6d0a55e7 修改基础镜像 node 版本 2024-03-25 22:42:28 +08:00
whyour 8ab2dc3280 修复重启命令 2024-03-22 23:01:20 +08:00
whyour 3f54048127 修改压缩解压文件命令 2024-03-22 22:23:49 +08:00
whyour 47c2c61f33 增加自定义通知 header 和 body 环境变量填写提示 2024-03-16 14:50:38 +08:00
whyour 47d2fc24bc 修改系统 reload 分支 2024-03-16 13:25:42 +08:00
whyour 7a8a8ab9b3 修改系统重启逻辑 2024-03-14 17:37:52 +08:00
whyour 2ac4581d54 修改自定义通知 body 解析逻辑 2024-03-12 22:22:10 +08:00
whyour 68ad01e0e8 增加 update 服务 2024-03-10 22:07:06 +08:00
180 changed files with 21723 additions and 11400 deletions
+8 -11
View File
@@ -1,14 +1,11 @@
UPDATE_PORT=5300
PUBLIC_PORT=5400
CRON_PORT=5500
BACK_PORT=5600
PORT=5700
GRPC_PORT=5500
BACK_PORT=5700
LOG_LEVEL='debug'
LOG_LEVEL='info'
SECRET='whyour'
JWT_SECRET=
JWT_EXPIRES_IN=
QINIU_AK=''
QINIU_SK=''
QINIU_SCOPE=''
TEMP=''
QINIU_AK=
QINIU_SK=
QINIU_SCOPE=
+4
View File
@@ -0,0 +1,4 @@
---
name: Bug Fixer
description: Fix this issue following our error handling pattern.
---
+272
View File
@@ -0,0 +1,272 @@
name: Build And Push Docker Image
on:
push:
paths-ignore:
- "*.md"
branches:
- "master"
- "develop"
tags:
- "v*"
workflow_dispatch:
jobs:
code_gitlab:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: Yikun/hub-mirror-action@master
with:
src: github/whyour
dst: gitlab/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITLAB_TOKEN }}
static_list: "qinglong"
force_update: true
code_gitee:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: Yikun/hub-mirror-action@master
with:
src: github/whyour
dst: gitee/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITEE_TOKEN }}
static_list: "qinglong"
force_update: true
build-static:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v4
with:
version: "8.3.1"
- uses: actions/setup-node@v6
with:
cache: "pnpm"
- name: build front and back
run: |
pnpm install --frozen-lockfile
pnpm build:front
pnpm build:back
- name: copy to static repo
env:
GITHUB_REPO: github.com/${{ github.repository_owner }}/qinglong-static
GITHUB_BRANCH: ${{ github.ref_name }}
REPO_GITEE: git@gitee.com:whyour/qinglong-static.git
REPO_GITLAB: git@gitlab.com:whyour/qinglong-static.git
PRIVATE_KEY: ${{ secrets.GITLAB_SSH_PK }}
run: |
mkdir -p tmp
cd ./tmp
cp -rf ../static/* ./
git init -b ${GITHUB_BRANCH} && git add .
git config --local user.name 'github-actions[bot]'
git config --local user.email 'github-actions[bot]@users.noreply.github.com'
git commit --allow-empty -m "copy static at $(date +'%Y-%m-%d %H:%M:%S')"
git push --force --quiet "https://${{ secrets.API_TOKEN }}@${GITHUB_REPO}.git" ${GITHUB_BRANCH}:${GITHUB_BRANCH}
static_gitlab:
needs: build-static
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: Yikun/hub-mirror-action@master
with:
src: github/whyour
dst: gitlab/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITLAB_TOKEN }}
static_list: "qinglong-static"
force_update: true
static_gitee:
needs: build-static
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: Yikun/hub-mirror-action@master
with:
src: github/whyour
dst: gitee/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITEE_TOKEN }}
static_list: "qinglong-static"
force_update: true
build:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
needs: build-static
runs-on: ubuntu-22.04
permissions:
packages: write
contents: read
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v4
with:
version: "8.3.1"
- uses: actions/setup-node@v6
with:
cache: "pnpm"
- name: Read version from version.yaml
id: version
run: |
VERSION=$(grep '^version:' version.yaml | awk '{print $2}')
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "Version: $VERSION"
- name: Setup timezone
uses: szenius/set-timezone@v2.0
with:
timezoneLinux: Asia/Shanghai
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ github.repository }}
ghcr.io/${{ github.repository }}
flavor: |
latest=false
tags: |
type=ref,event=branch,enable=${{ github.ref == format('refs/heads/{0}', 'develop') }}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=raw,value=${{ steps.version.outputs.version }},enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=semver,pattern={{version}}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push
id: docker_build
uses: docker/build-push-action@v6
with:
build-args: |
MAINTAINER=${{ github.repository_owner }}
QL_BRANCH=${{ github.ref_name }}
SOURCE_COMMIT=${{ github.sha }}
network: host
# linux/s390x npm 暂不可用
platforms: linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/386
context: .
file: ./docker/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=whyour/qinglong:cache
cache-to: type=registry,ref=whyour/qinglong:cache,mode=max
- name: Image digest
run: |
echo ${{ steps.docker_build.outputs.digest }}
build310:
if: ${{ github.ref_name == 'master' }}
needs: build-static
runs-on: ubuntu-22.04
permissions:
packages: write
contents: read
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v4
with:
version: "8.3.1"
- uses: actions/setup-node@v6
with:
cache: "pnpm"
- name: Read version from version.yaml
id: version
run: |
VERSION=$(grep '^version:' version.yaml | awk '{print $2}')
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "Version: $VERSION"
- name: Setup timezone
uses: szenius/set-timezone@v2.0
with:
timezoneLinux: Asia/Shanghai
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push python3.10
id: docker_build_310
uses: docker/build-push-action@v6
with:
build-args: |
MAINTAINER=${{ github.repository_owner }}
QL_BRANCH=${{ github.ref_name }}
SOURCE_COMMIT=${{ github.sha }}
network: host
# linux/s390x npm 暂不可用
platforms: linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/386
context: .
file: ./docker/310.Dockerfile
push: true
tags: |
whyour/qinglong:python3.10
whyour/qinglong:${{ steps.version.outputs.version }}-python3.10
cache-from: type=registry,ref=whyour/qinglong:cache-python3.10
cache-to: type=registry,ref=whyour/qinglong:cache-python3.10,mode=max
- name: Image digest
run: |
echo ${{ steps.docker_build_310.outputs.digest }}
-163
View File
@@ -1,163 +0,0 @@
name: Build And Push Docker Image
on:
push:
branches:
- "npm-debug"
workflow_dispatch:
jobs:
to_gitlab:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: pixta-dev/repository-mirroring-action@v1
with:
target_repo_url: git@gitlab.com:whyour/qinglong.git
ssh_private_key: ${{ secrets.GITLAB_SSH_PK }}
to_gitee:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: pixta-dev/repository-mirroring-action@v1
with:
target_repo_url: git@gitee.com:whyour/qinglong.git
ssh_private_key: ${{ secrets.GITLAB_SSH_PK }}
build-static:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
with:
version: "8.3.1"
- uses: actions/setup-node@v4
with:
cache: "pnpm"
- name: build front and back
run: |
pnpm install --frozen-lockfile
pnpm build:front
pnpm build:back
- name: copy to static repo
env:
GITHUB_REPO: github.com/${{ github.repository_owner }}/qinglong-static
GITHUB_BRANCH: ${{ github.ref_name }}
REPO_GITEE: git@gitee.com:whyour/qinglong-static.git
REPO_GITLAB: git@gitlab.com:whyour/qinglong-static.git
PRIVATE_KEY: ${{ secrets.GITLAB_SSH_PK }}
run: |
mkdir -p tmp
cd ./tmp
cp -rf ../static/* ./
git init -b ${GITHUB_BRANCH} && git add .
git config --local user.name 'github-actions[bot]'
git config --local user.email 'github-actions[bot]@users.noreply.github.com'
git commit --allow-empty -m "copy static at $(date +'%Y-%m-%d %H:%M:%S')"
git push --force --quiet "https://${{ secrets.API_TOKEN }}@${GITHUB_REPO}.git" ${GITHUB_BRANCH}:${GITHUB_BRANCH}
mkdir -p ~/.ssh
echo "${PRIVATE_KEY}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
export GIT_SSH_COMMAND="ssh -v -i ~/.ssh/id_rsa -o StrictHostKeyChecking=no -l git"
git remote add gitee "${REPO_GITEE}"
git remote add gitlab "${REPO_GITLAB}"
git gc
git push --force --quiet gitee ${GITHUB_BRANCH}:${GITHUB_BRANCH}
git push --force --quiet gitlab ${GITHUB_BRANCH}:${GITHUB_BRANCH}
build:
needs: build-static
# 由于 ubuntu-latest 中使用 linux6.x 内核 linux/s390x npm 无法使用
runs-on: ubuntu-20.04
# runs-on: self-hosted
permissions:
packages: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
with:
version: "8.3.1"
- uses: actions/setup-node@v4
with:
cache: "pnpm"
- name: Setup timezone
uses: szenius/set-timezone@v1.2
with:
timezoneLinux: Asia/Shanghai
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ github.repository }}
ghcr.io/${{ github.repository }}
# generate Docker tags based on the following events/attributes
# nightly, master, pr-2, 1.2.3, 1.2, 1
flavor: |
latest=false
tags: |
type=schedule,pattern=nightly
type=edge
type=ref,event=pr
type=ref,event=branch,enable=${{ github.ref != format('refs/heads/{0}', 'master') }}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
build-args: |
MAINTAINER=${{ github.repository_owner }}
QL_BRANCH=${{ github.ref_name }}
SOURCE_COMMIT=${{ github.sha }}
network: host
platforms: linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x,linux/386
# platforms: linux/amd64,linux/arm64,linux/ppc64le,linux/s390x,linux/386
context: .
file: ./docker/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Image digest
run: |
echo ${{ steps.docker_build.outputs.digest }}
+6 -1
View File
@@ -22,4 +22,9 @@
.env
.history
.version.ts
/.tmp
/.tmp
__pycache__
/shell/preload/env.*
/shell/preload/notify.*
/shell/preload/*-notify.json
/shell/preload/__ql_notify__.*
+11 -12
View File
@@ -2,23 +2,22 @@
**/*.svg
**/*.ejs
**/*.html
package.json
.umi
.umi-production
.umi-test
.history
.tmp
node_modules
/.umi
/.umi-production
/.umi-test
/.history
/.tmp
/node_modules
npm-debug.log*
yarn-error.log
yarn.lock
package-lock.json
static
data
/static
/data
DS_Store
src/.umi
src/.umi-production
src/.umi-test
/src/.umi
/src/.umi-production
/src/.umi-test
.env.local
.env
version.ts
+3 -20
View File
@@ -16,18 +16,8 @@ export default defineConfig({
favicons: [`https://qn.whyour.cn/favicon.svg`],
publicPath: process.env.NODE_ENV === 'production' ? './' : '/',
proxy: {
[`${baseUrl}api/update`]: {
target: 'http://127.0.0.1:5300/',
changeOrigin: true,
pathRewrite: { [`^${baseUrl}api/update`]: '/api' },
},
[`${baseUrl}api/public`]: {
target: 'http://127.0.0.1:5400/',
changeOrigin: true,
pathRewrite: { [`^${baseUrl}api/public`]: '/api' },
},
[`${baseUrl}api`]: {
target: 'http://127.0.0.1:5600/',
target: 'http://127.0.0.1:5700/',
changeOrigin: true,
ws: true,
pathRewrite: { [`^${baseUrl}api`]: '/api' },
@@ -43,19 +33,12 @@ export default defineConfig({
}),
);
}) as any,
externals: {
react: 'window.React',
'react-dom': 'window.ReactDOM',
},
headScripts: [
`./api/env.js`,
'https://gw.alipayobjects.com/os/lib/react/18.2.0/umd/react.production.min.js',
'https://gw.alipayobjects.com/os/lib/react-dom/18.2.0/umd/react-dom.production.min.js',
],
headScripts: [`./api/env.js`],
copy: [
{
from: 'node_modules/monaco-editor/min/vs',
to: 'static/dist/monaco-editor/min/vs',
},
],
npmClient: 'pnpm',
});
+16 -131
View File
@@ -18,9 +18,9 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
[docker-image-size-image]: https://img.shields.io/docker/image-size/whyour/qinglong?style=flat
[docker-image-size-url]: https://hub.docker.com/r/whyour/qinglong
[Demo](http://demo.dlww.cc:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[Demo](http://demo.qinglong.online:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[演示](http://demo.dlww.cc:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
[演示](http://demo.qinglong.online:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
</div>
![cover](https://user-images.githubusercontent.com/22700758/244847235-8dc1ca21-e03f-4606-9458-0541fab60413.png)
@@ -41,6 +41,8 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
The `latest` image is built on `alpine` and the `debian` image is built on `debian-slim`. If you need to use a dependency that is not supported by `alpine`, it is recommended that you use the `debian` image.
**⚠️ Important**: If you need to run Docker as a **non-root user**, please use the `debian` image. Alpine's `crond` requires root privileges.
```bash
docker pull whyour/qinglong:latest
docker pull whyour/qinglong:debian
@@ -48,151 +50,34 @@ docker pull whyour/qinglong:debian
### npm
The npm version supports `debian/ubuntu/centos/alpine` systems and requires `node/python3` to be installed.
The npm version supports `debian/ubuntu/alpine` systems and requires `node/npm/python3/pip3/pnpm` to be installed.
```bash
npm i @whyour/qinglong
```
## Built-in commands
- task
```bash
# Execute in sequence, if a random delay is set, it will be randomly delayed by a certain number of seconds
task <file_path>
# Execute in sequence, regardless of whether a random delay is set, all run immediately,
# and the foreground will output the day, while recorded in the log file
task <file_path> now
# Concurrent execution, regardless of whether a random delay is set, are run immediately,
# the foreground does not generate the day, directly recorded in the log file, and can be specified account execution
task <file_path> conc <env_name> <account_number>(Optional)
# Specify the account to execute and run immediately regardless of whether a random delay is set
task <file_path> desi <env_name> <account_number>
# Set task timeout
task -m <max_time> <file_path>
# Use -- to split, -- followed by a parameter that is passed to the script, as in the following example, the script receives the parameter -u whyour -p password
task <file_path> -- -u whyour -p password
```
- ql
```bash
# Update and restart Green Dragon
ql update
# Run custom scripts extra.sh
ql extra
# Adding a single script file
ql raw <file_url>
# Add a specific script for a single repository
ql repo <repo_url> <whitelist> <blacklist> <dependence> <branch>
# Delete old logs
ql rmlog <days>
# Start bot
ql bot
# Detecting the Green Dragon environment and repairing it
ql check
# Reset the number of login errors
ql resetlet
# Disable two-step login
ql resettfa
```
| **Parameter** | **Description** |
|---|---|
| file_url | Script address |
| repo_url | Repository address |
| whitelist | The whitelist when pulling the repository, i.e., the string contained in the path of the script to be pulled |
| blacklist | Blacklisting when pulling repositories, i.e. strings that are not included in the path of the script to be pulled |
| dependence | Pulling the dependencies needed for the repository will be copied directly from the repository to the repository directory under scripts, regardless of the blacklist |
| extensions | Pull the branch of the repository |
| branch | Number of days of logs to be kept |
| days | File path for task execution |
| file_path | The name of the environment variable that needs to be concurrent or specified at the time of task execution |
## Deployment
### Docker (Recommended)
[View Documentation](https://qinglong.online/guide/getting-started/installation-guide)
```bash
# curl -sSL get.docker.com | sh
docker run -dit \
-v $PWD/ql/data:/ql/data \
# The 5700 after the colon is the default port, if QlPort is set, it needs to be the same as QlPort.
-p 5700:5700 \
# Deployment paths are not required, e.g. /test.
-e QlBaseUrl="/" \
# Deployment port is not required, when using host mode, you can set the port after service startup, default 5700
-e QlPort="5700" \
--name qinglong \
--hostname qinglong \
--restart unless-stopped \
whyour/qinglong:latest
```
## Built-in API
### Docker-compose (Recommended)
[View Documentation](https://qinglong.online/guide/user-guide/built-in-api)
```bash
# curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
mkdir qinglong
wget https://raw.githubusercontent.com/whyour/qinglong/master/docker/docker-compose.yml
## Built-in commands
# start
docker-compose up -d
# stop
docker-compose down
```
### Podman (Recommended)
```bash
# https://podman.io/getting-started/installation
podman run -dit \
--network bridge \
-v $PWD/ql/data:/ql/data \
# The 5700 after the colon is the default port, if QlPort is set, it needs to be the same as QlPort.
-p 5700:5700 \
# Deployment paths are not required, e.g. /test.
-e QlBaseUrl="/" \
# Deployment port is not required, when using host mode, you can set the port after service startup, default 5700
-e QlPort="5700" \
--name qinglong \
--hostname qinglong \
docker.io/whyour/qinglong:latest
```
### Local
It is recommended to use a pure system installation to avoid losing the original system data, you need to install node/npm/python3/pip3 yourself
```bash
# Debian/Ubuntu
curl -sL https://deb.nodesource.com/setup_20.x | sudo -E bash -
# Centos
curl --silent --location https://rpm.nodesource.com/setup_20.x | sudo bash
```
```bash
npm install -g node-pre-gyp pnpm@8.3.1
npm install -g @whyour/qinglong
qinglong
# Add the environment variables QL_DIR and QL_DATA_DIR when prompted
export QL_DIR=""
export QL_DATA_DIR=""
# Run again
qinglong
```
[View Documentation](https://qinglong.online/guide/user-guide/basic-explanation)
## Development
```bash
$ git clone https://github.com/whyour/qinglong.git
$ cd qinglong
$ cp .env.example .env
git clone https://github.com/whyour/qinglong.git
cd qinglong
cp .env.example .env
# Recommended use pnpm https://pnpm.io/zh/installation
$ npm install -g pnpm@8.3.1
$ pnpm install
$ pnpm start
npm install -g pnpm@8.3.1
pnpm install
pnpm start
```
Open your browser and visit <http://127.0.0.1:5700>
+16 -129
View File
@@ -20,9 +20,9 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
[docker-image-size-image]: https://img.shields.io/docker/image-size/whyour/qinglong?style=flat
[docker-image-size-url]: https://hub.docker.com/r/whyour/qinglong
[Demo](http://demo.dlww.cc:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[Demo](http://demo.qinglong.online:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[演示](http://demo.dlww.cc:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
[演示](http://demo.qinglong.online:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
</div>
![cover](https://user-images.githubusercontent.com/22700758/244847235-8dc1ca21-e03f-4606-9458-0541fab60413.png)
@@ -43,6 +43,8 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
`latest` 镜像是基于 `alpine` 构建,`debian` 镜像是基于 `debian-slim` 构建。如果需要使用 `alpine` 不支持的依赖,建议使用 `debian` 镜像
**⚠️ 重要提示**: 如果您需要以**非 root 用户**运行 Docker,请使用 `debian` 镜像。Alpine 的 `crond` 需要 root 权限。
```bash
docker pull whyour/qinglong:latest
docker pull whyour/qinglong:debian
@@ -50,149 +52,34 @@ docker pull whyour/qinglong:debian
### npm
npm 版本支持 `debian/ubuntu/centos/alpine` 系统,需要自行安装 `node/python3`
npm 版本支持 `debian/ubuntu/alpine` 系统,需要自行安装 `node/npm/python3/pip3/pnpm`
```bash
npm i @whyour/qinglong
```
## 内置命令
- task
```bash
# 依次执行,如果设置了随机延迟,将随机延迟一定秒数
task <file_path>
# 依次执行,无论是否设置了随机延迟,均立即运行,前台会输出日,同时记录在日志文件中
task <file_path> now
# 并发执行,无论是否设置了随机延迟,均立即运行,前台不产生日,直接记录在日志文件中,且可指定账号执行
task <file_path> conc <env_name> <account_number>(可选的)
# 指定账号执行,无论是否设置了随机延迟,均立即运行
task <file_path> desi <env_name> <account_number>
# 设置任务超时时间
task -m <max_time> <file_path>
# 使用 -- 分割,-- 后面的参数会传给脚本,下面的例子,脚本就可接收到参数 -u whyour -p password
task <file_path> -- -u whyour -p password
```
- ql
```bash
# 更新并重启青龙
ql update
# 运行自定义脚本extra.sh
ql extra
# 添加单个脚本文件
ql raw <file_url>
# 添加单个仓库的指定脚本
ql repo <repo_url> <whitelist> <blacklist> <dependence> <branch> <extensions>
# 删除旧日志
ql rmlog <days>
# 启动tg-bot
ql bot
# 检测青龙环境并修复
ql check
# 重置登录错误次数
ql resetlet
# 禁用两步登录
ql resettfa
```
| **参数** | **说明** |
|------------|---------------------------------------------------------------------------------------------|
| file_url | 脚本地址 |
| repo_url | 仓库地址 |
| whitelist | 拉取仓库时的白名单,即就是需要拉取的脚本的路径包含的字符串,多个竖线分割 |
| blacklist | 拉取仓库时的黑名单,即就是需要拉取的脚本的路径不包含的字符串,多个竖线分割 |
| dependence | 拉取仓库需要的依赖文件,会直接从仓库拷贝到scripts下的仓库目录,不受黑名单影响,多个竖线分割 |
| extensions | 拉取仓库的文件后缀,多个竖线分割 |
| branch | 拉取仓库的分支 |
| days | 需要保留的日志的天数 |
| file_path | 任务执行时的文件路径 |
## 部署
### docker (推荐)
[查看文档](https://qinglong.online/guide/getting-started/installation-guide)
```bash
# curl -sSL get.docker.com | sh
docker run -dit \
-v $PWD/ql/data:/ql/data \
# 冒号后面的 5700 为默认端口,如果设置了 QlPort, 需要跟 QlPort 保持一致
-p 5700:5700 \
# 部署路径非必须,比如 /test
-e QlBaseUrl="/" \
# 部署端口非必须,当使用 host 模式时,可以设置服务启动后的端口,默认 5700
-e QlPort="5700" \
--name qinglong \
--hostname qinglong \
--restart unless-stopped \
whyour/qinglong:latest
```
## 内置 API
### docker-compose (推荐)
[查看文档](https://qinglong.online/guide/user-guide/built-in-api)
```bash
# curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
mkdir qinglong
wget https://raw.githubusercontent.com/whyour/qinglong/master/docker/docker-compose.yml
## 内置命令
# 启动
docker-compose up -d
# 停止
docker-compose down
```
### podman (推荐)
```bash
# https://podman.io/getting-started/installation
podman run -dit \
--network bridge \
-v $PWD/ql/data:/ql/data \
# 冒号后面的 5700 为默认端口,如果设置了 QlPort, 需要跟 QlPort 保持一致
-p 5700:5700 \
# 部署路径非必须,比如 /test
-e QlBaseUrl="/" \
# 部署端口非必须,当使用 host 模式时,可以设置服务启动后的端口,默认 5700
-e QlPort="5700" \
--name qinglong \
--hostname qinglong \
docker.io/whyour/qinglong:latest
```
### 本机
建议使用纯净系统安装,避免系统原有数据丢失,需要自己安装 node/npm/python3/pip3
```bash
# Debian/Ubuntu
curl -sL https://deb.nodesource.com/setup_20.x | sudo -E bash -
# Centos
curl --silent --location https://rpm.nodesource.com/setup_20.x | sudo bash
```
```bash
npm install -g node-pre-gyp pnpm@8.3.1
npm install -g @whyour/qinglong
qinglong
# 根据提示增加环境变量 QL_DIR 和 QL_DATA_DIR
export QL_DIR=""
export QL_DATA_DIR=""
# 再次执行
qinglong
```
[查看文档](https://qinglong.online/guide/user-guide/basic-explanation)
## 开发
```bash
$ git clone https://github.com/whyour/qinglong.git
$ cd qinglong
$ cp .env.example .env
git clone https://github.com/whyour/qinglong.git
cd qinglong
cp .env.example .env
# 推荐使用 pnpm https://pnpm.io/zh/installation
$ npm install -g pnpm@8.3.1
$ pnpm install
$ pnpm start
npm install -g pnpm@8.3.1
pnpm install
pnpm start
```
打开你的浏览器,访问 <http://127.0.0.1:5700>
+5
View File
@@ -0,0 +1,5 @@
## Reporting a Vulnerability
To report a vulnerability, please open a private vulnerability report at <https://github.com/whyour/qinglong/security>.
While the discovery of new vulnerabilities is rare, we also recommend always using the latest versions of Qinglong to ensure your application remains as secure as possible.
+2 -2
View File
@@ -1,4 +1,3 @@
import { getFileContentByName, getLastModifyFilePath } from '../config/util';
import { Router, Request, Response, NextFunction } from 'express';
import { Container } from 'typedi';
import { Logger } from 'winston';
@@ -8,6 +7,7 @@ import { celebrate, Joi } from 'celebrate';
import { join } from 'path';
import { SAMPLE_FILES } from '../config/const';
import ConfigService from '../services/config';
import { writeFileWithLock } from '../shared/utils';
const route = Router();
export default (app: Router) => {
@@ -78,7 +78,7 @@ export default (app: Router) => {
if (name.startsWith('data/scripts/')) {
path = join(config.rootPath, name);
}
await fs.writeFile(path, content);
await writeFileWithLock(path, content);
res.send({ code: 200, message: '保存成功' });
} catch (e) {
return next(e);
+11 -37
View File
@@ -4,7 +4,8 @@ import { Logger } from 'winston';
import CronService from '../services/cron';
import CronViewService from '../services/cronView';
import { celebrate, Joi } from 'celebrate';
import cron_parser from 'cron-parser';
import { commonCronSchema } from '../validation/schedule';
const route = Router();
export default (app: Router) => {
@@ -170,27 +171,14 @@ export default (app: Router) => {
route.post(
'/',
celebrate({
body: Joi.object({
command: Joi.string().required(),
schedule: Joi.string().required(),
name: Joi.string().optional(),
labels: Joi.array().optional(),
sub_id: Joi.number().optional().allow(null),
extra_schedules: Joi.array().optional().allow(null),
task_before: Joi.string().optional().allow('').allow(null),
task_after: Joi.string().optional().allow('').allow(null),
}),
body: Joi.object(commonCronSchema),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
if (cron_parser.parseExpression(req.body.schedule).hasNext()) {
const cronService = Container.get(CronService);
const data = await cronService.create(req.body);
return res.send({ code: 200, data });
} else {
return res.send({ code: 400, message: 'param schedule error' });
}
const cronService = Container.get(CronService);
const data = await cronService.create(req.body);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
@@ -331,30 +319,16 @@ export default (app: Router) => {
'/',
celebrate({
body: Joi.object({
labels: Joi.array().optional().allow(null),
command: Joi.string().required(),
schedule: Joi.string().required(),
name: Joi.string().optional().allow(null),
sub_id: Joi.number().optional().allow(null),
extra_schedules: Joi.array().optional().allow(null),
task_before: Joi.string().optional().allow('').allow(null),
task_after: Joi.string().optional().allow('').allow(null),
...commonCronSchema,
id: Joi.number().required(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
if (
!req.body.schedule ||
cron_parser.parseExpression(req.body.schedule).hasNext()
) {
const cronService = Container.get(CronService);
const data = await cronService.update(req.body);
return res.send({ code: 200, data });
} else {
return res.send({ code: 400, message: 'param schedule error' });
}
const cronService = Container.get(CronService);
const data = await cronService.update(req.body);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
@@ -418,7 +392,7 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.import_crontab();
const data = await cronService.importCrontab();
return res.send({ code: 200, data });
} catch (e) {
return next(e);
+22 -17
View File
@@ -8,17 +8,28 @@ const route = Router();
export default (app: Router) => {
app.use('/dependencies', route);
route.get('/', async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.dependencies(req.query as any);
return res.send({ code: 200, data });
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
});
route.get(
'/',
celebrate({
query:
Joi.object({
searchValue: Joi.string().optional().allow(''),
type: Joi.string().optional().allow(''),
status: Joi.string().optional().allow(''),
}).unknown(true),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.dependencies(req.query as any);
return res.send({ code: 200, data });
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
},
);
route.post(
'/',
@@ -32,7 +43,6 @@ export default (app: Router) => {
),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.create(req.body);
@@ -54,7 +64,6 @@ export default (app: Router) => {
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.update(req.body);
@@ -71,7 +80,6 @@ export default (app: Router) => {
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.remove(req.body);
@@ -88,7 +96,6 @@ export default (app: Router) => {
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.remove(req.body, true);
@@ -107,7 +114,6 @@ export default (app: Router) => {
}),
}),
async (req: Request<{ id: number }>, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.getDb({ id: req.params.id });
@@ -124,7 +130,6 @@ export default (app: Router) => {
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.reInstall(req.body);
+45 -8
View File
@@ -1,12 +1,12 @@
import { Router, Request, Response, NextFunction } from 'express';
import { Container } from 'typedi';
import EnvService from '../services/env';
import { Logger } from 'winston';
import { celebrate, Joi } from 'celebrate';
import multer from 'multer';
import config from '../config';
import { Joi, celebrate } from 'celebrate';
import { NextFunction, Request, Response, Router } from 'express';
import fs from 'fs';
import multer from 'multer';
import { Container } from 'typedi';
import { Logger } from 'winston';
import config from '../config';
import { safeJSONParse } from '../config/util';
import EnvService from '../services/env';
const route = Router();
const storage = multer.diskStorage({
@@ -51,6 +51,9 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
if (!req.body?.length) {
return res.send({ code: 400, message: '参数不正确' });
}
const data = await envService.create(req.body);
return res.send({ code: 200, data });
} catch (e) {
@@ -193,6 +196,40 @@ export default (app: Router) => {
},
);
route.put(
'/pin',
celebrate({
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.pin(req.body);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
},
);
route.put(
'/unpin',
celebrate({
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.unPin(req.body);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
},
);
route.post(
'/upload',
upload.single('env'),
@@ -217,7 +254,7 @@ export default (app: Router) => {
} else {
return res.send({
code: 400,
message: '文件缺少name或者value字段,参考导出文件格式',
message: '每条数据 name 或者 value 字段不能为空,参考导出文件格式',
});
}
} catch (e) {
+27
View File
@@ -0,0 +1,27 @@
import { Router } from 'express';
import Logger from '../loaders/logger';
import { HealthService } from '../services/health';
import Container from 'typedi';
const route = Router();
export default (app: Router) => {
app.use('/', route);
route.get('/health', async (req, res) => {
try {
const healthService = Container.get(HealthService);
const health = await healthService.check();
res.status(200).send({
code: 200,
data: health,
});
} catch (err: any) {
Logger.error('Health check failed:', err);
res.status(500).send({
code: 500,
message: 'Health check failed',
error: err.message,
});
}
});
};
+4
View File
@@ -9,6 +9,8 @@ import open from './open';
import dependence from './dependence';
import system from './system';
import subscription from './subscription';
import update from './update';
import health from './health';
export default () => {
const app = Router();
@@ -22,6 +24,8 @@ export default () => {
dependence(app);
system(app);
subscription(app);
update(app);
health(app);
return app;
};
+75 -26
View File
@@ -1,11 +1,15 @@
import { Router, Request, Response, NextFunction } from 'express';
import { celebrate, Joi } from 'celebrate';
import { NextFunction, Request, Response, Router } from 'express';
import { Container } from 'typedi';
import { Logger } from 'winston';
import * as fs from 'fs';
import config from '../config';
import { getFileContentByName, readDirs, rmPath } from '../config/util';
import { join } from 'path';
import { celebrate, Joi } from 'celebrate';
import {
getFileContentByName,
readDirs,
removeAnsi,
rmPath,
} from '../config/util';
import LogService from '../services/log';
const route = Router();
const blacklist = ['.tmp'];
@@ -30,16 +34,19 @@ export default (app: Router) => {
'/detail',
async (req: Request, res: Response, next: NextFunction) => {
try {
if (blacklist.includes(req.path)) {
return res.send({ code: 403, message: '暂无权限' });
}
const filePath = join(
config.logPath,
(req.query.path || '') as string,
req.query.file as string,
const logService = Container.get(LogService);
const finalPath = logService.checkFilePath(
(req.query.path as string) || '',
(req.query.file as string) || '',
);
const content = await getFileContentByName(filePath);
res.send({ code: 200, data: content });
if (!finalPath || blacklist.includes(req.query.path as string)) {
return res.send({
code: 403,
message: '暂无权限',
});
}
const content = await getFileContentByName(finalPath);
res.send({ code: 200, data: removeAnsi(content) });
} catch (e) {
return next(e);
}
@@ -50,15 +57,18 @@ export default (app: Router) => {
'/:file',
async (req: Request, res: Response, next: NextFunction) => {
try {
if (blacklist.includes(req.path)) {
return res.send({ code: 403, message: '暂无权限' });
}
const filePath = join(
config.logPath,
(req.query.path || '') as string,
req.params.file,
const logService = Container.get(LogService);
const finalPath = logService.checkFilePath(
(req.query.path as string) || '',
(req.params.file as string) || '',
);
const content = await getFileContentByName(filePath);
if (!finalPath || blacklist.includes(req.query.path as string)) {
return res.send({
code: 403,
message: '暂无权限',
});
}
const content = await getFileContentByName(finalPath);
res.send({ code: 200, data: content });
} catch (e) {
return next(e);
@@ -77,17 +87,56 @@ export default (app: Router) => {
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path, type } = req.body as {
let { filename, path } = req.body as {
filename: string;
path: string;
type: string;
};
const filePath = join(config.logPath, path, filename);
await rmPath(filePath);
const logService = Container.get(LogService);
const finalPath = logService.checkFilePath(path, filename);
if (!finalPath || blacklist.includes(path)) {
return res.send({
code: 403,
message: '暂无权限',
});
}
await rmPath(finalPath);
res.send({ code: 200 });
} catch (e) {
return next(e);
}
},
);
route.post(
'/download',
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path } = req.body as {
filename: string;
path: string;
};
const logService = Container.get(LogService);
const filePath = logService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
return res.download(filePath, filename, (err) => {
if (err) {
return next(err);
}
});
} catch (e) {
return next(e);
}
},
);
};
+130 -77
View File
@@ -1,11 +1,4 @@
import {
fileExist,
getFileContentByName,
readDirs,
getLastModifyFilePath,
readDir,
rmPath,
} from '../config/util';
import { fileExist, readDirs, readDir, rmPath, IFile } from '../config/util';
import { Router, Request, Response, NextFunction } from 'express';
import { Container } from 'typedi';
import { Logger } from 'winston';
@@ -15,6 +8,7 @@ import { celebrate, Joi } from 'celebrate';
import path, { join, parse } from 'path';
import ScriptService from '../services/script';
import multer from 'multer';
import { writeFileWithLock } from '../shared/utils';
const route = Router();
const storage = multer.diskStorage({
@@ -30,55 +24,68 @@ const upload = multer({ storage: storage });
export default (app: Router) => {
app.use('/scripts', route);
route.get('/', async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let result = [];
const blacklist = [
'node_modules',
'.git',
'.pnpm',
'pnpm-lock.yaml',
'yarn.lock',
'package-lock.json',
];
if (req.query.path) {
const targetPath = path.join(
config.scriptPath,
req.query.path as string,
);
result = await readDir(targetPath, config.scriptPath, blacklist);
} else {
result = await readDirs(
config.scriptPath,
config.scriptPath,
blacklist,
(a, b) => {
if (a.type === b.type) {
return a.title.localeCompare(b.title);
} else {
return a.type === 'directory' ? -1 : 1;
}
},
);
route.get(
'/',
celebrate({
query: Joi.object({
path: Joi.string().optional().allow(''),
}).unknown(true),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let result: IFile[] = [];
const blacklist = [
'node_modules',
'.git',
'.pnpm',
'pnpm-lock.yaml',
'yarn.lock',
'package-lock.json',
];
if (req.query.path) {
result = await readDir(
req.query.path as string,
config.scriptPath,
blacklist,
);
} else {
result = await readDirs(
config.scriptPath,
config.scriptPath,
blacklist,
(a, b) => {
if (a.type === b.type) {
return a.title.localeCompare(b.title);
} else {
return a.type === 'directory' ? -1 : 1;
}
},
);
}
res.send({
code: 200,
data: result,
});
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
res.send({
code: 200,
data: result,
});
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
});
});
route.get(
'/detail',
celebrate({
query: Joi.object({
path: Joi.string().optional().allow(''),
file: Joi.string().required(),
}).unknown(true),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const scriptService = Container.get(ScriptService);
const content = await scriptService.getFile(
req.query.path as string,
req.query?.path as string || '',
req.query.file as string,
);
res.send({ code: 200, data: content });
@@ -90,11 +97,19 @@ export default (app: Router) => {
route.get(
'/:file',
celebrate({
params: Joi.object({
file: Joi.string().required(),
}),
query: Joi.object({
path: Joi.string().optional().allow(''),
}).unknown(true),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const scriptService = Container.get(ScriptService);
const content = await scriptService.getFile(
req.query.path as string,
req.query?.path as string || '',
req.params.file,
);
res.send({ code: 200, data: content });
@@ -107,8 +122,17 @@ export default (app: Router) => {
route.post(
'/',
upload.single('file'),
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().optional().allow(''),
content: Joi.string().optional().allow(''),
originFilename: Joi.string().optional().allow(''),
directory: Joi.string().optional().allow(''),
file: Joi.string().optional().allow(''),
}).unknown(true),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename, path, content, originFilename, directory } =
req.body as {
@@ -130,13 +154,13 @@ export default (app: Router) => {
}
if (config.writePathList.every((x) => !path.startsWith(x))) {
return res.send({
code: 430,
message: '文件路径禁止访问',
code: 403,
message: '暂无权限',
});
}
if (req.file) {
await fs.rename(req.file.path, join(path, req.file.filename));
await fs.rename(req.file.path, join(path, filename));
return res.send({ code: 200 });
}
@@ -152,6 +176,7 @@ export default (app: Router) => {
path,
`${originFilename.replace(/\//g, '')}`,
);
await fs.mkdir(path, { recursive: true });
const filePath = join(path, `${filename.replace(/\//g, '')}`);
const fileExists = await fileExist(filePath);
if (fileExists) {
@@ -163,7 +188,7 @@ export default (app: Router) => {
await rmPath(originFilePath);
}
}
await fs.writeFile(filePath, content);
await writeFileWithLock(filePath, content);
return res.send({ code: 200 });
} catch (e) {
return next(e);
@@ -181,15 +206,21 @@ export default (app: Router) => {
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename, content, path } = req.body as {
filename: string;
content: string;
path: string;
};
const filePath = join(config.scriptPath, path, filename);
await fs.writeFile(filePath, content);
const scriptService = Container.get(ScriptService);
const filePath = scriptService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
await writeFileWithLock(filePath, content);
return res.send({ code: 200 });
} catch (e) {
return next(e);
@@ -202,19 +233,27 @@ export default (app: Router) => {
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().allow(''),
path: Joi.string().optional().allow(''),
type: Joi.string().optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename, path, type } = req.body as {
let { filename, path } = req.body as {
filename: string;
path: string;
type: string;
};
const filePath = join(config.scriptPath, path, filename);
if (!path) {
path = '';
}
const scriptService = Container.get(ScriptService);
const filePath = scriptService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
await rmPath(filePath);
res.send({ code: 200 });
} catch (e) {
@@ -228,24 +267,30 @@ export default (app: Router) => {
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().optional().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename } = req.body as {
let { filename, path } = req.body as {
filename: string;
path: string;
};
const filePath = join(config.scriptPath, filename);
// const stats = fs.statSync(filePath);
// res.set({
// 'Content-Type': 'application/octet-stream', //告诉浏览器这是一个二进制文件
// 'Content-Disposition': 'attachment; filename=' + filename, //告诉浏览器这是一个需要下载的文件
// 'Content-Length': stats.size //文件大小
// });
// fs.createReadStream(filePath).pipe(res);
if (!path) {
path = '';
}
const scriptService = Container.get(ScriptService);
const filePath = scriptService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
return res.download(filePath, filename, (err) => {
return next(err);
if (err) {
return next(err);
}
});
} catch (e) {
return next(e);
@@ -266,9 +311,12 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
let { filename, content, path } = req.body;
if (!path) {
path = '';
}
const { name, ext } = parse(filename);
const filePath = join(config.scriptPath, path, `${name}.swap${ext}`);
await fs.writeFile(filePath, content || '', { encoding: 'utf8' });
await writeFileWithLock(filePath, content || '');
const scriptService = Container.get(ScriptService);
const result = await scriptService.runScript(filePath);
@@ -291,6 +339,9 @@ export default (app: Router) => {
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path, pid } = req.body;
if (!path) {
path = '';
}
const { name, ext } = parse(filename);
const filePath = join(config.scriptPath, path, `${name}.swap${ext}`);
const logPath = join(config.logPath, path, `${name}.swap`);
@@ -318,12 +369,14 @@ export default (app: Router) => {
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path, type, newFilename } = req.body as {
let { filename, path, newFilename } = req.body as {
filename: string;
path: string;
type: string;
newFilename: string;
};
if (!path) {
path = '';
}
const filePath = join(config.scriptPath, path, filename);
const newPath = join(config.scriptPath, path, newFilename);
await fs.rename(filePath, newPath);
+6 -5
View File
@@ -3,7 +3,7 @@ import { Container } from 'typedi';
import { Logger } from 'winston';
import SubscriptionService from '../services/subscription';
import { celebrate, Joi } from 'celebrate';
import cron_parser from 'cron-parser';
import CronExpressionParser from 'cron-parser';
const route = Router();
export default (app: Router) => {
@@ -15,6 +15,7 @@ export default (app: Router) => {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.list(
req.query.searchValue as string,
req.query.ids as string,
);
return res.send({ code: 200, data });
} catch (e) {
@@ -59,7 +60,7 @@ export default (app: Router) => {
try {
if (
!req.body.schedule ||
cron_parser.parseExpression(req.body.schedule).hasNext()
CronExpressionParser.parse(req.body.schedule).hasNext()
) {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.create(req.body);
@@ -192,7 +193,7 @@ export default (app: Router) => {
if (
!req.body.schedule ||
typeof req.body.schedule === 'object' ||
cron_parser.parseExpression(req.body.schedule).hasNext()
CronExpressionParser.parse(req.body.schedule).hasNext()
) {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.update(req.body);
@@ -212,8 +213,8 @@ export default (app: Router) => {
body: Joi.array().items(Joi.number().required()),
query: Joi.object({
force: Joi.boolean().optional(),
t: Joi.number()
})
t: Joi.number(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
+127 -14
View File
@@ -14,6 +14,7 @@ import {
} from '../config/util';
import dayjs from 'dayjs';
import multer from 'multer';
import { logStreamManager } from '../shared/logStreamManager';
const route = Router();
const storage = multer.diskStorage({
@@ -33,7 +34,7 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const userService = Container.get(UserService);
const authInfo = await userService.getUserInfo();
const authInfo = await userService.getAuthInfo();
const { version, changeLog, changeLogLink, publishTime } =
await parseVersion(config.versionFile);
@@ -273,19 +274,22 @@ export default (app: Router) => {
{
onStart: async (cp, startTime) => {
res.setHeader('QL-Task-Pid', `${cp.pid}`);
res.setHeader('QL-Task-Log', `${logPath}`);
},
onEnd: async (cp, endTime, diff) => {
// Close the stream after task completion
await logStreamManager.closeStream(await handleLogPath(logPath));
res.end();
},
onError: async (message: string) => {
res.write(`\n${message}`);
res.write(message);
const absolutePath = await handleLogPath(logPath);
await fs.appendFile(absolutePath, `\n${message}`);
await logStreamManager.write(absolutePath, message);
},
onLog: async (message: string) => {
res.write(`\n${message}`);
res.write(message);
const absolutePath = await handleLogPath(logPath);
await fs.appendFile(absolutePath, `\n${message}`);
await logStreamManager.write(absolutePath, message);
},
},
);
@@ -316,10 +320,15 @@ export default (app: Router) => {
route.put(
'/data/export',
celebrate({
body: Joi.object({
type: Joi.array().items(Joi.string()).optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
await systemService.exportData(res);
await systemService.exportData(res, req.body.type);
} catch (e) {
return next(e);
}
@@ -340,12 +349,116 @@ export default (app: Router) => {
},
);
route.get('/log', async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
await systemService.getSystemLog(res);
} catch (e) {
return next(e);
}
});
route.get(
'/log',
celebrate({
query: {
startTime: Joi.string().allow('').optional(),
endTime: Joi.string().allow('').optional(),
t: Joi.string().optional(),
},
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
await systemService.getSystemLog(
res,
req.query as {
startTime?: string;
endTime?: string;
},
);
} catch (e) {
return next(e);
}
},
);
route.delete(
'/log',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
await systemService.deleteSystemLog();
res.send({ code: 200 });
} catch (e) {
return next(e);
}
},
);
route.put(
'/auth/reset',
celebrate({
body: Joi.object({
retries: Joi.number().optional(),
twoFactorActivated: Joi.boolean().optional(),
password: Joi.string().optional(),
username: Joi.string().optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const userService = Container.get(UserService);
await userService.resetAuthInfo(req.body);
res.send({ code: 200, message: '更新成功' });
} catch (e) {
return next(e);
}
},
);
route.put(
'/config/timezone',
celebrate({
body: Joi.object({
timezone: Joi.string().allow('').allow(null),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.updateTimezone(req.body);
res.send(result);
} catch (e) {
return next(e);
}
},
);
route.put(
'/config/global-ssh-key',
celebrate({
body: Joi.object({
globalSshKey: Joi.string().allow('').allow(null),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.updateGlobalSshKey(req.body);
res.send(result);
} catch (e) {
return next(e);
}
},
);
route.put(
'/config/dependence-clean',
celebrate({
body: Joi.object({
type: Joi.string().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.cleanDependence(req.body.type);
res.send(result);
} catch (e) {
return next(e);
}
},
);
};
+51
View File
@@ -0,0 +1,51 @@
import { NextFunction, Request, Response, Router } from 'express';
import Container from 'typedi';
import Logger from '../loaders/logger';
import SystemService from '../services/system';
const route = Router();
export default (app: Router) => {
app.use('/update', route);
route.put(
'/reload',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem();
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
route.put(
'/system',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('system');
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
route.put(
'/data',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('data');
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
};
+5 -4
View File
@@ -8,6 +8,7 @@ import path from 'path';
import { v4 as uuidV4 } from 'uuid';
import rateLimit from 'express-rate-limit';
import config from '../config';
import { isDemoEnv, getToken } from '../config/util';
const route = Router();
const storage = multer.diskStorage({
@@ -55,7 +56,8 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const userService = Container.get(UserService);
await userService.logout(req.platform);
const token = getToken(req);
await userService.logout(req.platform, token);
res.send({ code: 200 });
} catch (e) {
return next(e);
@@ -72,9 +74,8 @@ export default (app: Router) => {
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
if (process.env.DeployEnv === 'demo') {
if (isDemoEnv()) {
return res.send({ code: 450, message: '未知错误' });
}
const userService = Container.get(UserService);
@@ -90,7 +91,7 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const userService = Container.get(UserService);
const authInfo = await userService.getUserInfo();
const authInfo = await userService.getAuthInfo();
res.send({
code: 200,
data: {
+303 -27
View File
@@ -1,32 +1,308 @@
import 'reflect-metadata'; // We need this in order to use @Decorators
import config from './config';
import 'reflect-metadata';
import cluster, { type Worker } from 'cluster';
import compression from 'compression';
import cors from 'cors';
import express from 'express';
import helmet from 'helmet';
import { Container } from 'typedi';
import config from './config';
import Logger from './loaders/logger';
import { monitoringMiddleware } from './middlewares/monitoring';
import { type GrpcServerService } from './services/grpc';
import { type HttpServerService } from './services/http';
async function startServer() {
const app = express();
await require('./loaders/db').default();
await require('./loaders/initFile').default();
await require('./loaders/sentry').default({ expressApp: app });
await require('./loaders/app').default({ expressApp: app });
const server = app
.listen(config.port, () => {
Logger.debug(`✌️ 后端服务启动成功!`);
console.debug(`✌️ 后端服务启动成功!`);
process.send?.('ready');
})
.on('error', (err) => {
Logger.error(err);
console.error(err);
process.exit(1);
});
await require('./loaders/server').default({ server });
interface WorkerMetadata {
id: number;
pid: number;
serviceType: string;
startTime: Date;
}
startServer();
class Application {
private app: express.Application;
private httpServerService?: HttpServerService;
private grpcServerService?: GrpcServerService;
private isShuttingDown = false;
private workerMetadataMap = new Map<number, WorkerMetadata>();
private httpWorker?: Worker;
constructor() {
this.app = express();
// 创建一个全局中间件,删除查询参数中的t
this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {
if (req.query.t) {
delete req.query.t;
}
next();
});
}
async start() {
try {
if (cluster.isPrimary) {
await this.initializeDatabase();
}
if (cluster.isPrimary) {
this.startMasterProcess();
} else {
await this.startWorkerProcess();
}
} catch (error) {
Logger.error('Failed to start application:', error);
process.exit(1);
}
}
private startMasterProcess() {
// Fork gRPC worker first and wait for it to be ready
const grpcWorker = this.forkWorker('grpc');
// Wait for gRPC worker to signal it's ready before starting HTTP worker
this.waitForWorkerReady(grpcWorker, 30000)
.then(() => {
Logger.info('✌️ gRPC worker is ready, starting HTTP worker');
this.httpWorker = this.forkWorker('http');
})
.catch((error) => {
Logger.error('✌️ Failed to wait for gRPC worker:', error);
process.exit(1);
});
cluster.on('exit', (worker, code, signal) => {
const metadata = this.workerMetadataMap.get(worker.id);
if (metadata) {
if (!this.isShuttingDown) {
Logger.error(
`✌️ ${metadata.serviceType} worker ${worker.process.pid} died (${signal || code
}). Restarting...`,
);
// If gRPC worker died, restart it and wait for it to be ready
if (metadata.serviceType === 'grpc') {
const newGrpcWorker = this.forkWorker('grpc');
this.waitForWorkerReady(newGrpcWorker, 30000)
.then(() => {
Logger.info('✌️ gRPC worker restarted and ready');
// Re-register cron jobs by notifying the HTTP worker
if (this.httpWorker) {
try {
this.httpWorker.send('reregister-crons');
Logger.info('✌️ Sent reregister-crons message to HTTP worker');
} catch (error) {
Logger.error('✌️ Failed to send reregister-crons message:', error);
}
}
})
.catch((error) => {
Logger.error('✌️ Failed to restart gRPC worker:', error);
process.exit(1);
});
} else {
// For HTTP worker, just restart it
const newWorker = this.forkWorker(metadata.serviceType);
this.httpWorker = newWorker;
Logger.info(`✌️ Restarted ${metadata.serviceType} worker (PID: ${newWorker.process.pid})`);
}
}
this.workerMetadataMap.delete(worker.id);
}
});
this.setupMasterShutdown();
}
private waitForWorkerReady(worker: Worker, timeoutMs: number): Promise<void> {
return new Promise<void>((resolve, reject) => {
const messageHandler = (msg: any) => {
if (msg === 'ready') {
worker.removeListener('message', messageHandler);
clearTimeout(timeoutId);
resolve();
}
};
worker.on('message', messageHandler);
// Timeout after specified milliseconds
const timeoutId = setTimeout(() => {
worker.removeListener('message', messageHandler);
reject(new Error(`Worker failed to start within ${timeoutMs / 1000} seconds`));
}, timeoutMs);
});
}
private forkWorker(serviceType: string): Worker {
const worker = cluster.fork({ SERVICE_TYPE: serviceType });
this.workerMetadataMap.set(worker.id, {
id: worker.id,
pid: worker.process.pid!,
serviceType,
startTime: new Date(),
});
return worker;
}
private async initializeDatabase() {
const dbLoader = await import('./loaders/db');
await dbLoader.default();
}
private setupMiddlewares() {
this.app.use(helmet({
contentSecurityPolicy: false,
crossOriginOpenerPolicy: false,
originAgentCluster: false,
}));
this.app.use(cors(config.cors));
this.app.use(compression());
this.app.use(monitoringMiddleware);
}
private setupMasterShutdown() {
const shutdown = async () => {
if (this.isShuttingDown) return;
this.isShuttingDown = true;
const workers = Object.values(cluster.workers || {});
const workerPromises: Promise<void>[] = [];
workers.forEach((worker) => {
if (worker) {
const exitPromise = new Promise<void>((resolve) => {
worker.once('exit', () => {
Logger.info(`✌️ Worker ${worker.process.pid} exited`);
resolve();
});
try {
worker.send('shutdown');
} catch (error) {
Logger.warn(
`✌️ Failed to send shutdown to worker ${worker.process.pid}:`,
error,
);
}
});
workerPromises.push(exitPromise);
}
});
try {
await Promise.race([
Promise.all(workerPromises),
new Promise<void>((resolve) => {
setTimeout(() => {
Logger.warn('✌️ Worker shutdown timeout reached');
resolve();
}, 10000);
}),
]);
process.exit(0);
} catch (error) {
Logger.error('✌️ Error during worker shutdown:', error);
process.exit(1);
}
};
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
}
private async startWorkerProcess() {
const serviceType = process.env.SERVICE_TYPE;
if (!serviceType || !['http', 'grpc'].includes(serviceType)) {
Logger.error('✌️ Invalid SERVICE_TYPE:', serviceType);
process.exit(1);
}
Logger.info(`✌️ ${serviceType} worker started (PID: ${process.pid})`);
try {
if (serviceType === 'http') {
await this.startHttpService();
} else {
await this.startGrpcService();
}
process.send?.('ready');
} catch (error) {
Logger.error(`✌️ ${serviceType} worker failed:`, error);
process.exit(1);
}
}
private async startHttpService() {
this.setupMiddlewares();
const { HttpServerService } = await import('./services/http');
this.httpServerService = Container.get(HttpServerService);
const appLoader = await import('./loaders/app');
await appLoader.default({ app: this.app });
const server = await this.httpServerService.initialize(
this.app,
config.port,
);
const serverLoader = await import('./loaders/server');
await (serverLoader.default as any)({ server });
this.setupWorkerShutdown('http');
}
private async startGrpcService() {
const { GrpcServerService } = await import('./services/grpc');
this.grpcServerService = Container.get(GrpcServerService);
await this.grpcServerService.initialize();
this.setupWorkerShutdown('grpc');
}
private setupWorkerShutdown(serviceType: string) {
process.on('message', async (msg) => {
if (msg === 'shutdown') {
this.gracefulShutdown(serviceType);
} else if (msg === 'reregister-crons' && serviceType === 'http') {
// Re-register cron jobs when gRPC worker restarts
try {
Logger.info('✌️ Received reregister-crons message, re-registering cron jobs...');
const CronService = (await import('./services/cron')).default;
const cronService = Container.get(CronService);
await cronService.autosave_crontab();
Logger.info('✌️ Cron jobs re-registered successfully');
} catch (error) {
Logger.error('✌️ Failed to re-register cron jobs:', error);
}
}
});
const shutdown = () => this.gracefulShutdown(serviceType);
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
}
private async gracefulShutdown(serviceType: string) {
if (this.isShuttingDown) return;
this.isShuttingDown = true;
try {
if (serviceType === 'http') {
await this.httpServerService?.shutdown();
} else {
await this.grpcServerService?.shutdown();
}
process.exit(0);
} catch (error) {
Logger.error(`✌️ [${serviceType}] Error during shutdown:`, error);
process.exit(1);
}
}
}
const app = new Application();
app.start().catch((error) => {
Logger.error('🙅‍♀️ Application failed to start:', error);
process.exit(1);
});
+26
View File
@@ -23,3 +23,29 @@ export const SAMPLE_FILES = [
target: 'data/scripts/notify.py',
},
];
export const PYTHON_INSTALL_DIR = process.env.PYTHON_HOME;
export const NotificationModeStringMap = {
0: 'gotify',
1: 'goCqHttpBot',
2: 'serverChan',
3: 'pushDeer',
4: 'bark',
5: 'chat',
6: 'telegramBot',
7: 'dingtalkBot',
8: 'weWorkBot',
9: 'weWorkApp',
10: 'aibotk',
11: 'iGot',
12: 'pushPlus',
13: 'wePlusBot',
14: 'email',
15: 'pushMe',
16: 'feishu',
17: 'webhook',
18: 'chronocat',
19: 'ntfy',
20: 'wxPusherBot',
} as const;
+68
View File
@@ -0,0 +1,68 @@
import { request as undiciRequest, Dispatcher } from 'undici';
type RequestBaseOptions = {
dispatcher?: Dispatcher;
json?: Record<string, any>;
form?: string;
headers?: Record<string, string>;
} & Omit<Dispatcher.RequestOptions<null>, 'origin' | 'path' | 'method'>;
type RequestOptionsWithOptions = RequestBaseOptions &
Partial<Pick<Dispatcher.RequestOptions, 'method'>>;
type ResponseTypeMap = {
json: Record<string, any>;
text: string;
};
type ResponseTypeKey = keyof ResponseTypeMap;
async function request(
url: string,
options?: RequestOptionsWithOptions,
): Promise<Dispatcher.ResponseData<null>> {
const { json, form, body, headers = {}, ...rest } = options || {};
const finalHeaders = { ...headers } as Record<string, string>;
let finalBody = body;
if (json) {
finalHeaders['content-type'] = 'application/json';
finalBody = JSON.stringify(json);
} else if (form) {
finalBody = form;
delete finalHeaders['content-type'];
}
const res = await undiciRequest(url, {
method: 'POST',
headers: finalHeaders,
body: finalBody,
...rest,
});
return res;
}
async function post<T extends ResponseTypeKey = 'json'>(
url: string,
options?: RequestBaseOptions & { responseType?: T },
): Promise<ResponseTypeMap[T]> {
const resp = await request(url, { ...options, method: 'POST' });
const rawText = await resp.body.text();
if (options?.responseType === 'text') {
return rawText as ResponseTypeMap[T];
}
try {
return JSON.parse(rawText) as ResponseTypeMap[T];
} catch {
return rawText as ResponseTypeMap[T];
}
}
export const httpClient = {
post,
request,
};
+90 -15
View File
@@ -2,12 +2,60 @@ import dotenv from 'dotenv';
import path from 'path';
import { createRandomString } from './share';
dotenv.config({
path: path.join(__dirname, '../../.env'),
});
interface Config {
port: number;
grpcPort: number;
nodeEnv: string;
isDevelopment: boolean;
isProduction: boolean;
jwt: {
secret: string;
expiresIn?: string;
};
cors: {
origin: string[];
methods: string[];
};
logs: {
level: string;
};
api: {
prefix: string;
};
}
const config: Config = {
port: parseInt(process.env.BACK_PORT || '5700', 10),
grpcPort: parseInt(process.env.GRPC_PORT || '5500', 10),
nodeEnv: process.env.NODE_ENV || 'development',
isDevelopment: process.env.NODE_ENV === 'development',
isProduction: process.env.NODE_ENV === 'production',
logs: {
level: process.env.LOG_LEVEL || 'silly',
},
api: {
prefix: '/api',
},
jwt: {
secret: process.env.JWT_SECRET || 'whyour-secret',
expiresIn: process.env.JWT_EXPIRES_IN,
},
cors: {
origin: process.env.CORS_ORIGIN
? process.env.CORS_ORIGIN.split(',')
: ['*'],
methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
},
};
process.env.NODE_ENV = process.env.NODE_ENV || 'development';
if (!process.env.QL_DIR) {
// 声明QL_DIR环境变量
let qlHomePath = path.join(__dirname, '../../');
// 生产环境
if (qlHomePath.endsWith('/static/')) {
qlHomePath = path.join(qlHomePath, '../');
}
@@ -16,23 +64,48 @@ if (!process.env.QL_DIR) {
const lastVersionFile = `https://qn.whyour.cn/version.yaml`;
// Get and normalize QlBaseUrl
let baseUrl = process.env.QlBaseUrl || '';
if (baseUrl) {
// Ensure it starts with /
if (!baseUrl.startsWith('/')) {
baseUrl = `/${baseUrl}`;
}
// Remove trailing slash for consistency in route definitions
if (baseUrl.endsWith('/')) {
baseUrl = baseUrl.slice(0, -1);
}
}
const rootPath = process.env.QL_DIR as string;
const envFound = dotenv.config({ path: path.join(rootPath, '.env') });
const dataPath = path.join(rootPath, 'data/');
let dataPath = path.join(rootPath, 'data/');
if (process.env.QL_DATA_DIR) {
dataPath = process.env.QL_DATA_DIR.replace(/\/$/g, '');
}
const shellPath = path.join(rootPath, 'shell/');
const preloadPath = path.join(shellPath, 'preload/');
const tmpPath = path.join(rootPath, '.tmp/');
const samplePath = path.join(rootPath, 'sample/');
const configPath = path.join(dataPath, 'config/');
const scriptPath = path.join(dataPath, 'scripts/');
const repoPath = path.join(dataPath, 'repo/');
const bakPath = path.join(dataPath, 'bak/');
const logPath = path.join(dataPath, 'log/');
const dbPath = path.join(dataPath, 'db/');
const uploadPath = path.join(dataPath, 'upload/');
const sshdPath = path.join(dataPath, 'ssh.d/');
const systemLogPath = path.join(dataPath, 'syslog/');
const dependenceCachePath = path.join(dataPath, 'dep_cache/');
const envFile = path.join(configPath, 'env.sh');
const envFile = path.join(preloadPath, 'env.sh');
const jsEnvFile = path.join(preloadPath, 'env.js');
const pyEnvFile = path.join(preloadPath, 'env.py');
const jsNotifyFile = path.join(preloadPath, '__ql_notify__.js');
const pyNotifyFile = path.join(preloadPath, '__ql_notify__.py');
const confFile = path.join(configPath, 'config.sh');
const crontabFile = path.join(configPath, 'crontab.list');
const authConfigFile = path.join(configPath, 'auth.json');
@@ -54,17 +127,9 @@ if (envFound.error) {
}
export default {
port: parseInt(process.env.BACK_PORT as string, 10),
cronPort: parseInt(process.env.CRON_PORT as string, 10),
publicPort: parseInt(process.env.PUBLIC_PORT as string, 10),
updatePort: parseInt(process.env.UPDATE_PORT as string, 10),
secret: process.env.SECRET || createRandomString(16, 32),
logs: {
level: process.env.LOG_LEVEL || 'silly',
},
api: {
prefix: '/api',
},
...config,
jwt: config.jwt,
baseUrl,
rootPath,
tmpPath,
dataPath,
@@ -82,10 +147,15 @@ export default {
sampleFile,
confFile,
envFile,
jsEnvFile,
pyEnvFile,
jsNotifyFile,
pyNotifyFile,
dbPath,
uploadPath,
configPath,
scriptPath,
repoPath,
samplePath,
blackFileList: [
'auth.json',
@@ -94,12 +164,15 @@ export default {
'crontab.list',
'dependence-proxy.sh',
'env.sh',
'env.js',
'env.py',
'token.json',
],
writePathList: [configPath, scriptPath],
bakPath,
apiWhiteList: [
'/api/user/login',
'/api/health',
'/open/auth/token',
'/api/user/two-factor/login',
'/api/system',
@@ -116,4 +189,6 @@ export default {
sqliteFile,
sshdPath,
systemLogPath,
dependenceCachePath,
maxTokensPerPlatform: 10, // Maximum number of concurrent sessions per platform
};
+3 -3
View File
@@ -5,13 +5,13 @@ let pickedEnv: Record<string, string>;
function getPickedEnv() {
if (pickedEnv) return pickedEnv;
const picked = pick(process.env, ['QlBaseUrl', 'DeployEnv']);
const picked = pick(process.env, ['QlBaseUrl', 'DeployEnv', 'QL_DIR']);
if (picked.QlBaseUrl) {
if (!picked.QlBaseUrl.startsWith('/')) {
picked.QlBaseUrl = `/${picked.QlBaseUrl}`
picked.QlBaseUrl = `/${picked.QlBaseUrl}`;
}
if (!picked.QlBaseUrl.endsWith('/')) {
picked.QlBaseUrl = `${picked.QlBaseUrl}/`
picked.QlBaseUrl = `${picked.QlBaseUrl}/`;
}
}
pickedEnv = picked as Record<string, string>;
+3 -1
View File
@@ -32,7 +32,9 @@ export function formatCommand(doc: Subscription, url?: string) {
autoDelCron,
} = doc;
if (type === 'file') {
command += `raw "${_url}"`;
command += `raw "${_url}" "${proxy || ''}" "${
isNil(autoAddCron) ? true : Boolean(autoAddCron)
}" "${isNil(autoDelCron) ? true : Boolean(autoDelCron)}"`;
} else {
command += `repo "${_url}" "${whitelist || ''}" "${blacklist || ''}" "${
dependences || ''
+188 -118
View File
@@ -1,15 +1,15 @@
import * as fs from 'fs/promises';
import * as path from 'path';
import got from 'got';
import iconv from 'iconv-lite';
import { exec } from 'child_process';
import FormData from 'form-data';
import psTreeFun from 'pstree.remy';
import psTreeFun from 'ps-tree';
import { promisify } from 'util';
import { load } from 'js-yaml';
import config from './index';
import { TASK_COMMAND } from './const';
import { PYTHON_INSTALL_DIR, TASK_COMMAND } from './const';
import Logger from '../loaders/logger';
import { writeFileWithLock } from '../shared/utils';
import { DependenceTypes } from '../data/dependence';
import { FormData } from 'undici';
export * from './share';
@@ -21,6 +21,10 @@ export async function getFileContentByName(fileName: string) {
return '';
}
export function removeAnsi(text: string) {
return text.replace(/\x1b\[\d+m/g, '');
}
export async function getLastModifyFilePath(dir: string) {
let filePath = '';
@@ -30,7 +34,7 @@ export async function getLastModifyFilePath(dir: string) {
arr.forEach(async (item) => {
const fullpath = path.join(dir, item);
const stats = await fs.stat(fullpath);
const stats = await fs.lstat(fullpath);
if (stats.isFile()) {
if (stats.mtimeMs >= 0) {
filePath = fullpath;
@@ -52,78 +56,6 @@ export function getToken(req: any) {
return '';
}
export async function getNetIp(req: any) {
const ipArray = [
...new Set([
...(req.headers['x-real-ip'] || '').split(','),
...(req.headers['x-forwarded-for'] || '').split(','),
req.ip,
...req.ips,
req.socket.remoteAddress,
]),
].filter(Boolean);
let ip = ipArray[0];
if (ipArray.length > 1) {
for (let i = 0; i < ipArray.length; i++) {
const ipNumArray = ipArray[i].split('.');
const tmp = ipNumArray[0] + '.' + ipNumArray[1];
if (
tmp === '192.168' ||
(ipNumArray[0] === '172' &&
ipNumArray[1] >= 16 &&
ipNumArray[1] <= 32) ||
tmp === '10.7' ||
tmp === '127.0'
) {
continue;
}
ip = ipArray[i];
break;
}
}
ip = ip.substr(ip.lastIndexOf(':') + 1, ip.length);
if (ip.includes('127.0') || ip.includes('192.168') || ip.includes('10.7')) {
ip = '';
}
if (!ip) {
return { address: `获取失败`, ip };
}
try {
const csdnApi = got
.get(`https://searchplugin.csdn.net/api/v1/ip/get?ip=${ip}`, {
timeout: 10000,
retry: 0,
})
.text();
const pconlineApi = got
.get(`https://whois.pconline.com.cn/ipJson.jsp?ip=${ip}&json=true`, {
timeout: 10000,
retry: 0,
})
.buffer();
const [csdnBody, pconlineBody] = await await Promise.all<any>([
csdnApi,
pconlineApi,
]);
const csdnRes = JSON.parse(csdnBody);
const pconlineRes = JSON.parse(iconv.decode(pconlineBody, 'GBK'));
let address = '';
if (csdnBody && csdnRes.code == 200) {
address = csdnRes.data.address;
} else if (pconlineRes && pconlineRes.addr) {
address = pconlineRes.addr;
}
return { address, ip };
} catch (error) {
return { address: `获取失败`, ip };
}
}
export function getPlatform(userAgent: string): 'mobile' | 'desktop' {
const ua = userAgent.toLowerCase();
const testUa = (regexp: RegExp) => regexp.test(ua);
@@ -145,12 +77,19 @@ export function getPlatform(userAgent: string): 'mobile' | 'desktop' {
system = 'android'; // android系统
} else if (testUa(/ios|iphone|ipad|ipod|iwatch/g)) {
system = 'ios'; // ios系统
} else if (testUa(/openharmony/g)) {
system = 'openharmony'; // openharmony系统
}
let platform = 'desktop';
if (system === 'windows' || system === 'macos' || system === 'linux') {
platform = 'desktop';
} else if (system === 'android' || system === 'ios' || testUa(/mobile/g)) {
} else if (
system === 'android' ||
system === 'ios' ||
system === 'openharmony' ||
testUa(/mobile/g)
) {
platform = 'mobile';
}
@@ -168,7 +107,7 @@ export async function fileExist(file: any) {
export async function createFile(file: string, data: string = '') {
await fs.mkdir(path.dirname(file), { recursive: true });
await fs.writeFile(file, data);
await writeFileWithLock(file, data);
}
export async function handleLogPath(
@@ -225,19 +164,19 @@ enum FileType {
'file',
}
interface IFile {
export interface IFile {
title: string;
key: string;
type: 'directory' | 'file';
parent: string;
mtime: number;
createTime: number;
size?: number;
children?: IFile[];
}
export function dirSort(a: IFile, b: IFile): number {
if (a.type === 'file' && b.type === 'file') {
return b.mtime - a.mtime;
return b.createTime - a.createTime;
} else if (a.type === 'directory' && b.type === 'directory') {
return a.title.localeCompare(b.title);
} else {
@@ -257,7 +196,7 @@ export async function readDirs(
for (const file of files) {
const subPath = path.join(dir, file);
const stats = await fs.stat(subPath);
const stats = await fs.lstat(subPath);
const key = path.join(relativePath, file);
if (blacklist.includes(file) || stats.isSymbolicLink()) {
@@ -271,7 +210,7 @@ export async function readDirs(
key,
type: 'directory',
parent: relativePath,
mtime: stats.mtime.getTime(),
createTime: stats.birthtime.getTime(),
children: children.sort(sort),
});
} else {
@@ -281,7 +220,7 @@ export async function readDirs(
key,
parent: relativePath,
size: stats.size,
mtime: stats.mtime.getTime(),
createTime: stats.birthtime.getTime(),
});
}
}
@@ -293,23 +232,51 @@ export async function readDir(
dir: string,
baseDir: string = '',
blacklist: string[] = [],
) {
const relativePath = path.relative(baseDir, dir);
const files = await fs.readdir(dir);
const result: any = files
.filter((x) => !blacklist.includes(x))
.map(async (file: string) => {
const subPath = path.join(dir, file);
const stats = await fs.stat(subPath);
): Promise<IFile[]> {
const absoluteDir = path.join(baseDir, dir);
const relativePath = path.relative(baseDir, absoluteDir);
try {
const files = await fs.readdir(absoluteDir);
const result: IFile[] = [];
for (const file of files) {
const subPath = path.join(absoluteDir, file);
const stats = await fs.lstat(subPath);
const key = path.join(relativePath, file);
return {
title: file,
type: stats.isDirectory() ? 'directory' : 'file',
key,
parent: relativePath,
};
});
return result;
if (blacklist.includes(file) || stats.isSymbolicLink()) {
continue;
}
if (stats.isDirectory()) {
result.push({
title: file,
type: 'directory',
key,
parent: relativePath,
createTime: stats.birthtime.getTime(),
children: [],
});
} else {
result.push({
title: file,
type: 'file',
key,
parent: relativePath,
size: stats.size,
createTime: stats.birthtime.getTime(),
});
}
}
return result;
} catch (error: any) {
if (error.code === 'ENOENT') {
return [];
}
throw error;
}
}
export async function promiseExec(command: string): Promise<string> {
@@ -340,9 +307,9 @@ export function parseHeaders(headers: string) {
if (!headers) return {};
const parsed: any = {};
let key;
let val;
let i;
let key: string;
let val: string;
let i: number;
headers &&
headers.split('\n').forEach(function parser(line) {
@@ -360,7 +327,10 @@ export function parseHeaders(headers: string) {
return parsed;
}
function parseString(input: string): Record<string, string> {
function parseString(
input: string,
valueFormatFn?: (v: string) => string,
): Record<string, string> {
const regex = /(\w+):\s*((?:(?!\n\w+:).)*)/g;
const matches: Record<string, string> = {};
@@ -372,9 +342,10 @@ function parseString(input: string): Record<string, string> {
continue;
}
const _value = value.trim();
let _value = value.trim();
try {
_value = valueFormatFn ? valueFormatFn(_value) : _value;
const jsonValue = JSON.parse(_value);
matches[_key] = jsonValue;
} catch (error) {
@@ -392,12 +363,13 @@ export function parseBody(
| 'multipart/form-data'
| 'application/x-www-form-urlencoded'
| 'text/plain',
valueFormatFn?: (v: string) => string,
) {
if (contentType === 'text/plain' || !body) {
return body;
return valueFormatFn && body ? valueFormatFn(body) : body;
}
const parsed = parseString(body);
const parsed = parseString(body, valueFormatFn);
switch (contentType) {
case 'multipart/form-data':
@@ -416,11 +388,11 @@ export function parseBody(
export function psTree(pid: number): Promise<number[]> {
return new Promise((resolve, reject) => {
psTreeFun(pid, (err: any, pids: number[]) => {
psTreeFun(pid, (err: any, children) => {
if (err) {
reject(err);
}
resolve(pids.filter((x) => !isNaN(x)));
resolve(children.map((x) => Number(x.PID)).filter((x) => !isNaN(x)));
});
});
}
@@ -445,6 +417,27 @@ export async function getPid(cmd: string) {
return pid ? Number(pid) : undefined;
}
export async function getAllPids(cmd: string): Promise<number[]> {
const taskCommand = `ps -eo pid,command | grep "${cmd}" | grep -v grep | awk '{print $1}'`;
const pidsStr = await promiseExec(taskCommand);
if (!pidsStr) return [];
return pidsStr
.split('\n')
.map((p) => Number(p.trim()))
.filter((p) => !isNaN(p) && p > 0);
}
export async function killAllTasks(cmd: string): Promise<void> {
const pids = await getAllPids(cmd);
for (const pid of pids) {
try {
await killTask(pid);
} catch (error) {
// Ignore errors if process already terminated
}
}
}
interface IVersion {
version: string;
changeLogLink: string;
@@ -456,7 +449,7 @@ export async function parseVersion(path: string): Promise<IVersion> {
return load(await fs.readFile(path, 'utf8')) as IVersion;
}
export async function parseContentVersion(content: string): Promise<IVersion> {
export function parseContentVersion(content: string): IVersion {
return load(content) as IVersion;
}
@@ -464,13 +457,18 @@ export async function getUniqPath(
command: string,
id: string,
): Promise<string> {
let suffix = '';
if (/^\d+$/.test(id)) {
id = `_${id}`;
} else {
id = '';
suffix = `_${id}`;
}
let items = command.split(/ +/);
const maxTimeCommandIndex = items.findIndex((x) => x === '-m');
if (maxTimeCommandIndex !== -1) {
items = items.slice(maxTimeCommandIndex + 2);
}
const items = command.split(/ +/);
let str = items[0];
if (items[0] === TASK_COMMAND) {
str = items[1];
@@ -494,7 +492,7 @@ export async function getUniqPath(
str = `${tempStr}_${str.slice(slashIndex + 1)}`;
}
return `${str}${id}`;
return `${str}${suffix}`;
}
export function safeJSONParse(value?: string) {
@@ -505,7 +503,7 @@ export function safeJSONParse(value?: string) {
try {
return JSON.parse(value);
} catch (error) {
Logger.error('[JSON.parse失败]', error);
Logger.error('[safeJSONParse失败]', error);
return {};
}
}
@@ -520,3 +518,75 @@ export async function rmPath(path: string) {
Logger.error('[rmPath失败]', error);
}
}
export async function setSystemTimezone(timezone: string): Promise<boolean> {
try {
if (!(await fileExist(`/usr/share/zoneinfo/${timezone}`))) {
throw new Error('Invalid timezone');
}
await promiseExec(`ln -sf /usr/share/zoneinfo/${timezone} /etc/localtime`);
await promiseExec(`echo "${timezone}" > /etc/timezone`);
return true;
} catch (error) {
Logger.error('[setSystemTimezone失败]', error);
return false;
}
}
export function getGetCommand(type: DependenceTypes, name: string): string {
const baseCommands = {
[DependenceTypes.nodejs]: `pnpm ls -g | grep "${name}" | head -1`,
[DependenceTypes.python3]: `
python3 -c "exec('''
name='${name}'
try:
from importlib.metadata import version
print(version(name))
except:
import importlib.util as u
import importlib.metadata as m
spec=u.find_spec(name)
print(name if spec else '')
''')"`,
[DependenceTypes.linux]: `apk info -es ${name}`,
};
return baseCommands[type];
}
export function getInstallCommand(type: DependenceTypes, name: string): string {
const baseCommands = {
[DependenceTypes.nodejs]: 'pnpm add -g',
[DependenceTypes.python3]:
'pip3 install --disable-pip-version-check --root-user-action=ignore',
[DependenceTypes.linux]: 'apk add --no-check-certificate',
};
let command = baseCommands[type];
if (type === DependenceTypes.python3 && PYTHON_INSTALL_DIR) {
command = `${command} --prefix=${PYTHON_INSTALL_DIR}`;
}
return `${command} ${name.trim()}`;
}
export function getUninstallCommand(
type: DependenceTypes,
name: string,
): string {
const baseCommands = {
[DependenceTypes.nodejs]: 'pnpm remove -g',
[DependenceTypes.python3]:
'pip3 uninstall --disable-pip-version-check --root-user-action=ignore -y',
[DependenceTypes.linux]: 'apk del',
};
return `${baseCommands[type]} ${name.trim()}`;
}
export function isDemoEnv() {
return process.env.DeployEnv === 'demo';
}
+7 -1
View File
@@ -21,6 +21,8 @@ export class Crontab {
extra_schedules?: Array<{ schedule: string }>;
task_before?: string;
task_after?: string;
log_name?: string;
allow_multiple_instances?: 1 | 0;
constructor(options: Crontab) {
this.name = options.name;
@@ -45,6 +47,8 @@ export class Crontab {
this.extra_schedules = options.extra_schedules;
this.task_before = options.task_before;
this.task_after = options.task_after;
this.log_name = options.log_name;
this.allow_multiple_instances = options.allow_multiple_instances || 0;
}
}
@@ -55,7 +59,7 @@ export enum CrontabStatus {
'disabled',
}
export interface CronInstance extends Model<Crontab, Crontab>, Crontab { }
export interface CronInstance extends Model<Crontab, Crontab>, Crontab {}
export const CrontabModel = sequelize.define<CronInstance>('Crontab', {
name: {
unique: 'compositeIndex',
@@ -84,4 +88,6 @@ export const CrontabModel = sequelize.define<CronInstance>('Crontab', {
extra_schedules: DataTypes.JSON,
task_before: DataTypes.STRING,
task_after: DataTypes.STRING,
log_name: DataTypes.STRING,
allow_multiple_instances: DataTypes.NUMBER,
});
-18
View File
@@ -41,30 +41,12 @@ export enum DependenceTypes {
'linux',
}
export enum InstallDependenceCommandTypes {
'pnpm add -g',
'pip3 install --disable-pip-version-check --root-user-action=ignore',
'apk add --no-check-certificate',
}
export enum GetDependenceCommandTypes {
'pnpm ls -g ',
'pip3 show --disable-pip-version-check',
'apk info -es',
}
export enum versionDependenceCommandTypes {
'@',
'==',
'=',
}
export enum unInstallDependenceCommandTypes {
'pnpm remove -g',
'pip3 uninstall --disable-pip-version-check --root-user-action=ignore -y',
'apk del',
}
export interface DependenceInstance
extends Model<Dependence, Dependence>,
Dependence {}
+5 -2
View File
@@ -1,5 +1,5 @@
import { DataTypes, Model } from 'sequelize';
import { sequelize } from '.';
import { DataTypes, Model, ModelDefined } from 'sequelize';
export class Env {
value?: string;
@@ -9,6 +9,7 @@ export class Env {
position?: number;
name?: string;
remarks?: string;
isPinned?: 1 | 0;
constructor(options: Env) {
this.value = options.value;
@@ -20,7 +21,8 @@ export class Env {
this.timestamp = new Date().toString();
this.position = options.position;
this.name = options.name;
this.remarks = options.remarks;
this.remarks = options.remarks || '';
this.isPinned = options.isPinned || 0;
}
}
@@ -42,4 +44,5 @@ export const EnvModel = sequelize.define<EnvInstance>('Env', {
position: DataTypes.NUMBER,
name: { type: DataTypes.STRING, unique: 'compositeIndex' },
remarks: DataTypes.STRING,
isPinned: DataTypes.NUMBER,
});
+41 -8
View File
@@ -1,5 +1,3 @@
import { IncomingHttpHeaders } from 'http';
export enum NotificationMode {
'gotify' = 'gotify',
'goCqHttpBot' = 'goCqHttpBot',
@@ -14,11 +12,14 @@ export enum NotificationMode {
'aibotk' = 'aibotk',
'iGot' = 'iGot',
'pushPlus' = 'pushPlus',
'wePlusBot' = 'wePlusBot',
'email' = 'email',
'pushMe' = 'pushMe',
'feishu' = 'feishu',
'webhook' = 'webhook',
'chronocat' = 'Chronocat',
'ntfy' = 'ntfy',
'wxPusherBot' = 'wxPusherBot',
}
abstract class NotificationBaseInfo {
@@ -46,9 +47,8 @@ export class PushDeerNotification extends NotificationBaseInfo {
public pushDeerUrl = '';
}
export class ChatNotification extends NotificationBaseInfo {
public chatUrl = '';
public chatToken = '';
export class synologyChatNotification extends NotificationBaseInfo {
public synologyChatUrl = '';
}
export class BarkNotification extends NotificationBaseInfo {
@@ -58,6 +58,7 @@ export class BarkNotification extends NotificationBaseInfo {
public barkGroup = 'qinglong';
public barkLevel = 'active';
public barkUrl = '';
public barkArchive = '';
}
export class TelegramBotNotification extends NotificationBaseInfo {
@@ -97,16 +98,29 @@ export class IGotNotification extends NotificationBaseInfo {
export class PushPlusNotification extends NotificationBaseInfo {
public pushPlusToken = '';
public pushPlusUser = '';
public pushPlusTemplate = '';
public pushplusChannel = '';
public pushplusWebhook = '';
public pushplusCallbackUrl = '';
public pushplusTo = '';
}
export class WePlusBotNotification extends NotificationBaseInfo {
public wePlusBotToken = '';
public wePlusBotReceiver = '';
public wePlusBotVersion = '';
}
export class EmailNotification extends NotificationBaseInfo {
public emailService: string = '';
public emailUser: string = '';
public emailPass: string = '';
public emailTo: string = '';
}
export class PushMeNotification extends NotificationBaseInfo {
public pushMeKey: string = '';
public pushMeUrl: string = '';
}
export class ChronocatNotification extends NotificationBaseInfo {
@@ -128,6 +142,23 @@ export class WebhookNotification extends NotificationBaseInfo {
export class LarkNotification extends NotificationBaseInfo {
public larkKey = '';
public larkSecret = '';
}
export class NtfyNotification extends NotificationBaseInfo {
public ntfyUrl = '';
public ntfyTopic = '';
public ntfyPriority = '';
public ntfyToken = '';
public ntfyUsername = '';
public ntfyPassword = '';
public ntfyActions = '';
}
export class WxPusherBotNotification extends NotificationBaseInfo {
public wxPusherBotAppToken = '';
public wxPusherBotTopicIds = '';
public wxPusherBotUids = '';
}
export interface NotificationInfo
@@ -135,7 +166,7 @@ export interface NotificationInfo
GotifyNotification,
ServerChanNotification,
PushDeerNotification,
ChatNotification,
synologyChatNotification,
BarkNotification,
TelegramBotNotification,
DingtalkBotNotification,
@@ -144,9 +175,11 @@ export interface NotificationInfo
AibotkNotification,
IGotNotification,
PushPlusNotification,
WePlusBotNotification,
EmailNotification,
PushMeNotification,
WebhookNotification,
ChronocatNotification,
LarkNotification {}
LarkNotification,
NtfyNotification,
WxPusherBotNotification {}
+1 -1
View File
@@ -24,7 +24,7 @@ export interface AppToken {
expiration: number;
}
export type AppScope = 'envs' | 'crons' | 'configs' | 'scripts' | 'logs';
export type AppScope = 'envs' | 'crons' | 'configs' | 'scripts' | 'logs' | 'system';
export interface AppInstance extends Model<App, App>, App {}
export const AppModel = sequelize.define<AppInstance>('App', {
+39 -4
View File
@@ -2,13 +2,13 @@ import { sequelize } from '.';
import { DataTypes, Model, ModelDefined } from 'sequelize';
import { NotificationInfo } from './notify';
export class AuthInfo {
export class SystemInfo {
ip?: string;
type: AuthDataType;
info?: SystemModelInfo;
id?: number;
constructor(options: AuthInfo) {
constructor(options: SystemInfo) {
this.ip = options.ip;
this.info = options.info;
this.type = options.type;
@@ -27,6 +27,7 @@ export enum AuthDataType {
'notification' = 'notification',
'removeLogFrequency' = 'removeLogFrequency',
'systemConfig' = 'systemConfig',
'authConfig' = 'authConfig',
}
export interface SystemConfigInfo {
@@ -36,6 +37,8 @@ export interface SystemConfigInfo {
nodeMirror?: string;
pythonMirror?: string;
linuxMirror?: string;
timezone?: string;
globalSshKey?: string;
}
export interface LoginLogInfo {
@@ -46,11 +49,43 @@ export interface LoginLogInfo {
status?: LoginStatus;
}
export interface TokenInfo {
value: string;
timestamp: number;
ip: string;
address: string;
platform: string;
/**
* Token expiration time in seconds since Unix epoch.
* If undefined, the token uses JWT's built-in expiration.
*/
expiration?: number;
}
export interface AuthInfo {
username: string;
password: string;
retries: number;
lastlogon: number;
lastip: string;
lastaddr: string;
platform: string;
isTwoFactorChecking: boolean;
token: string;
tokens: Record<string, string | TokenInfo[]>;
twoFactorActivated: boolean;
twoFactorSecret: string;
avatar: string;
}
export type SystemModelInfo = SystemConfigInfo &
Partial<NotificationInfo> &
LoginLogInfo;
LoginLogInfo &
Partial<AuthInfo>;
export interface SystemInstance extends Model<AuthInfo, AuthInfo>, AuthInfo { }
export interface SystemInstance
extends Model<SystemInfo, SystemInfo>,
SystemInfo {}
export const SystemModel = sequelize.define<SystemInstance>('Auth', {
ip: DataTypes.STRING,
type: DataTypes.STRING,
-7
View File
@@ -1,7 +0,0 @@
declare namespace Express {
interface Request {
platform: 'desktop' | 'mobile';
}
}
declare module 'pstree.remy';
+13
View File
@@ -0,0 +1,13 @@
export enum ScheduleType {
BOOT = '@boot',
ONCE = '@once',
}
export type ScheduleValidator = (schedule?: string) => boolean;
export type CronSchedulerPayload = {
name: string;
id: string;
schedule: string;
command: string;
extra_schedules: Array<{ schedule: string }>;
};
+19 -25
View File
@@ -5,30 +5,24 @@ import initData from './initData';
import { Application } from 'express';
import linkDeps from './deps';
import initTask from './initTask';
import initFile from './initFile';
export default async ({ expressApp }: { expressApp: Application }) => {
try {
depInjectorLoader();
Logger.info('✌️ Dependency Injector loaded');
console.log('✌️ Dependency Injector loaded');
expressLoader({ app: expressApp });
Logger.info('✌️ Express loaded');
console.log('✌️ Express loaded');
await initData();
Logger.info('✌️ init data loaded');
console.log('✌️ init data loaded');
await linkDeps();
Logger.info('✌️ link deps loaded');
console.log('✌️ link deps loaded');
initTask();
Logger.info('✌️ init task loaded');
console.log('✌️ init task loaded');
} catch (error) {
Logger.error(`✌️ depInjectorLoader expressLoader initData linkDeps failed, ${error}`);
console.error(`✌️ depInjectorLoader expressLoader initData linkDeps failed ${error}`);
}
export default async ({ app }: { app: Application }) => {
depInjectorLoader();
Logger.info('✌️ Dependency loaded');
await linkDeps();
Logger.info('✌️ Link deps loaded');
initFile();
Logger.info('✌️ Init file loaded');
await initData();
Logger.info('✌️ Init data loaded');
initTask();
Logger.info('✌️ Init task loaded');
expressLoader({ app });
Logger.info('✌️ Express loaded');
};
+8
View File
@@ -0,0 +1,8 @@
import Container from 'typedi';
import CronService from '../services/cron';
export default async () => {
const cronService = Container.get(CronService);
await cronService.bootTask();
};
+31 -112
View File
@@ -1,15 +1,11 @@
import Logger from './logger';
import path from 'path';
import DataStore from 'nedb';
import { EnvModel } from '../data/env';
import { CrontabModel } from '../data/cron';
import { DependenceModel } from '../data/dependence';
import { AppModel } from '../data/open';
import { SystemModel } from '../data/system';
import { fileExist } from '../config/util';
import { SubscriptionModel } from '../data/subscription';
import { CrontabViewModel } from '../data/cronView';
import config from '../config';
import { sequelize } from '../data';
export default async () => {
@@ -23,118 +19,41 @@ export default async () => {
await CrontabViewModel.sync();
// 初始化新增字段
try {
await sequelize.query(
'alter table CrontabViews add column filterRelation VARCHAR(255)',
);
} catch (error) {}
try {
await sequelize.query(
'alter table Subscriptions add column proxy VARCHAR(255)',
);
} catch (error) {}
try {
await sequelize.query('alter table CrontabViews add column type NUMBER');
} catch (error) {}
try {
await sequelize.query(
'alter table Subscriptions add column autoAddCron NUMBER',
);
} catch (error) {}
try {
await sequelize.query(
'alter table Subscriptions add column autoDelCron NUMBER',
);
} catch (error) {}
try {
await sequelize.query('alter table Crontabs add column sub_id NUMBER');
} catch (error) { }
try {
await sequelize.query('alter table Crontabs add column extra_schedules JSON');
} catch (error) { }
try {
await sequelize.query('alter table Crontabs add column task_before TEXT');
} catch (error) { }
try {
await sequelize.query('alter table Crontabs add column task_after TEXT');
} catch (error) { }
const migrations = [
{
table: 'CrontabViews',
column: 'filterRelation',
type: 'VARCHAR(255)',
},
{ table: 'Subscriptions', column: 'proxy', type: 'VARCHAR(255)' },
{ table: 'CrontabViews', column: 'type', type: 'NUMBER' },
{ table: 'Subscriptions', column: 'autoAddCron', type: 'NUMBER' },
{ table: 'Subscriptions', column: 'autoDelCron', type: 'NUMBER' },
{ table: 'Crontabs', column: 'sub_id', type: 'NUMBER' },
{ table: 'Crontabs', column: 'extra_schedules', type: 'JSON' },
{ table: 'Crontabs', column: 'task_before', type: 'TEXT' },
{ table: 'Crontabs', column: 'task_after', type: 'TEXT' },
{ table: 'Crontabs', column: 'log_name', type: 'VARCHAR(255)' },
{
table: 'Crontabs',
column: 'allow_multiple_instances',
type: 'NUMBER',
},
{ table: 'Envs', column: 'isPinned', type: 'NUMBER' },
];
// 2.10-2.11 升级
const cronDbFile = path.join(config.rootPath, 'db/crontab.db');
const envDbFile = path.join(config.rootPath, 'db/env.db');
const appDbFile = path.join(config.rootPath, 'db/app.db');
const authDbFile = path.join(config.rootPath, 'db/auth.db');
const dependenceDbFile = path.join(config.rootPath, 'db/dependence.db');
const crondbExist = await fileExist(cronDbFile);
const dependenceDbExist = await fileExist(dependenceDbFile);
const envDbExist = await fileExist(envDbFile);
const appDbExist = await fileExist(appDbFile);
const authDbExist = await fileExist(authDbFile);
const cronCount = await CrontabModel.count();
const dependenceCount = await DependenceModel.count();
const envCount = await EnvModel.count();
const appCount = await AppModel.count();
const authCount = await SystemModel.count();
if (crondbExist && cronCount === 0) {
const cronDb = new DataStore({
filename: cronDbFile,
autoload: true,
});
cronDb.persistence.compactDatafile();
cronDb.find({}).exec(async (err, docs) => {
await CrontabModel.bulkCreate(docs, { ignoreDuplicates: true });
});
for (const migration of migrations) {
try {
await sequelize.query(
`alter table ${migration.table} add column ${migration.column} ${migration.type}`,
);
} catch (error) {
// Column already exists or other error, continue
}
}
if (dependenceDbExist && dependenceCount === 0) {
const dependenceDb = new DataStore({
filename: dependenceDbFile,
autoload: true,
});
dependenceDb.persistence.compactDatafile();
dependenceDb.find({}).exec(async (err, docs) => {
await DependenceModel.bulkCreate(docs, { ignoreDuplicates: true });
});
}
if (envDbExist && envCount === 0) {
const envDb = new DataStore({
filename: envDbFile,
autoload: true,
});
envDb.persistence.compactDatafile();
envDb.find({}).exec(async (err, docs) => {
await EnvModel.bulkCreate(docs, { ignoreDuplicates: true });
});
}
if (appDbExist && appCount === 0) {
const appDb = new DataStore({
filename: appDbFile,
autoload: true,
});
appDb.persistence.compactDatafile();
appDb.find({}).exec(async (err, docs) => {
await AppModel.bulkCreate(docs, { ignoreDuplicates: true });
});
}
if (authDbExist && authCount === 0) {
const authDb = new DataStore({
filename: authDbFile,
autoload: true,
});
authDb.persistence.compactDatafile();
authDb.find({}).exec(async (err, docs) => {
await SystemModel.bulkCreate(docs, { ignoreDuplicates: true });
});
}
console.log('✌️ DB loaded');
Logger.info('✌️ DB loaded');
} catch (error) {
console.error('✌️ DB load failed');
Logger.error(error);
Logger.error('✌️ DB load failed', error);
}
};
+29 -8
View File
@@ -1,8 +1,9 @@
import path from 'path';
import fs from 'fs/promises';
import os from 'os';
import chokidar from 'chokidar';
import config from '../config/index';
import { fileExist, promiseExec, rmPath } from '../config/util';
import Logger from './logger';
async function linkToNodeModule(src: string, dst?: string) {
const target = path.join(config.rootPath, 'node_modules', dst || src);
@@ -13,28 +14,48 @@ async function linkToNodeModule(src: string, dst?: string) {
if (!stats) {
await fs.symlink(source, target, 'dir');
}
} catch (error) {}
} catch (error) { }
}
async function linkCommand() {
const commandPath = await promiseExec('which node');
const commandDir = path.dirname(commandPath);
const homeDir = os.homedir();
let userBinDir = path.join(homeDir, 'bin');
try {
await fs.mkdir(userBinDir, { recursive: true });
await linkCommandToDir(userBinDir);
} catch (error) {
Logger.error('Linking command failed:', error);
}
}
async function linkCommandToDir(commandDir: string) {
const linkShell = [
{
src: 'update.sh',
dest: 'ql',
tmp: 'ql_tmp',
},
{
src: 'task.sh',
dest: 'task',
tmp: 'task_tmp',
},
];
for (const link of linkShell) {
const source = path.join(config.rootPath, 'shell', link.src);
const target = path.join(commandDir, link.dest);
await rmPath(target);
await fs.symlink(source, target);
const tmpTarget = path.join(commandDir, link.tmp);
try {
const stats = await fs.lstat(tmpTarget);
if (stats) {
await fs.unlink(tmpTarget);
}
} catch (error) { }
await fs.symlink(source, tmpTarget);
await fs.rename(tmpTarget, target);
}
}
@@ -49,6 +70,6 @@ export default async (src: string = 'deps') => {
});
watcher
.on('add', (path) => linkToNodeModule(src))
.on('change', (path) => linkToNodeModule(src));
.on('add', () => linkToNodeModule(src))
.on('change', () => linkToNodeModule(src));
};
+75 -45
View File
@@ -3,45 +3,60 @@ import bodyParser from 'body-parser';
import cors from 'cors';
import routes from '../api';
import config from '../config';
import jwt, { UnauthorizedError } from 'express-jwt';
import fs from 'fs/promises';
import { getPlatform, getToken, safeJSONParse } from '../config/util';
import Container from 'typedi';
import OpenService from '../services/open';
import { UnauthorizedError, expressjwt } from 'express-jwt';
import { getPlatform, getToken } from '../config/util';
import rewrite from 'express-urlrewrite';
import UserService from '../services/user';
import * as Sentry from '@sentry/node';
import { EnvModel } from '../data/env';
import { errors } from 'celebrate';
import { createProxyMiddleware } from 'http-proxy-middleware';
import { serveEnv } from '../config/serverEnv';
import Logger from './logger';
import { IKeyvStore, shareStore } from '../shared/store';
import { isValidToken } from '../shared/auth';
import path from 'path';
export default ({ app }: { app: Application }) => {
// Security: Enable strict routing to prevent case-insensitive path bypass
app.set('case sensitive routing', true);
app.set('strict routing', true);
app.set('trust proxy', 'loopback');
app.use(cors());
// Security: Path normalization middleware to prevent case variation attacks
app.use((req, res, next) => {
const originalPath = req.path;
const normalizedPath = originalPath.toLowerCase();
// Block requests with case variations on protected paths
if (originalPath !== normalizedPath &&
(normalizedPath.startsWith('/api/') || normalizedPath.startsWith('/open/'))) {
return res.status(400).json({
code: 400,
message: 'Invalid path format'
});
}
next();
});
// Rewrite URLs to strip baseUrl prefix if configured
// This allows the rest of the app to work without baseUrl awareness
if (config.baseUrl) {
app.use(rewrite(`${config.baseUrl}/*`, '/$1'));
}
app.get(`${config.api.prefix}/env.js`, serveEnv);
app.use(`${config.api.prefix}/static`, express.static(config.uploadPath));
app.use(
'/api/public',
createProxyMiddleware({
target: `http://0.0.0.0:${config.publicPort}/api`,
changeOrigin: true,
pathRewrite: { '/api/public': '' },
logProvider: () => Logger,
}),
);
app.use(bodyParser.json({ limit: '50mb' }));
app.use(bodyParser.urlencoded({ limit: '50mb', extended: true }));
const frontendPath = path.join(config.rootPath, 'static/dist');
app.use(express.static(frontendPath));
app.use(
jwt({
secret: config.secret,
expressjwt({
secret: config.jwt.secret,
algorithms: ['HS384'],
}).unless({
path: [...config.apiWhiteList, /^\/open\//],
path: [...config.apiWhiteList, /^(\/(?!api\/).*)$/i],
}),
);
@@ -55,14 +70,21 @@ export default ({ app }: { app: Application }) => {
return next();
});
app.use(async (req, res, next) => {
app.use(async (req: Request, res, next) => {
const pathLower = req.path.toLowerCase();
if (!['/open/', '/api/'].some((x) => pathLower.startsWith(x))) {
return next();
}
const headerToken = getToken(req);
if (req.path.startsWith('/open/')) {
const openService = Container.get(OpenService);
const doc = await openService.findTokenByValue(headerToken);
if (pathLower.startsWith('/open/')) {
const apps = await shareStore.getApps();
const doc = apps?.filter((x) =>
x.tokens?.find((y) => y.value === headerToken),
)?.[0];
if (doc && doc.tokens && doc.tokens.length > 0) {
const currentToken = doc.tokens.find((x) => x.value === headerToken);
const keyMatch = req.path.match(/\/open\/([a-z]+)\/*/);
const keyMatch = pathLower.match(/\/open\/([a-z]+)\/*/);
const key = keyMatch && keyMatch[1];
if (
doc.scopes.includes(key as any) &&
@@ -83,12 +105,9 @@ export default ({ app }: { app: Application }) => {
return next();
}
const data = await fs.readFile(config.authConfigFile, 'utf8');
if (data && headerToken) {
const { token = '', tokens = {} } = safeJSONParse(data);
if (headerToken === token || tokens[req.platform] === headerToken) {
return next();
}
const authInfo = await shareStore.getAuthInfo();
if (isValidToken(authInfo, headerToken, req.platform)) {
return next();
}
const errorCode = headerToken ? 'invalid_token' : 'credentials_required';
@@ -100,11 +119,19 @@ export default ({ app }: { app: Application }) => {
});
app.use(async (req, res, next) => {
if (!['/api/user/init', '/api/user/notification/init'].includes(req.path)) {
const pathLower = req.path.toLowerCase();
if (
![
'/api/user/init',
'/api/user/notification/init',
'/open/user/init',
'/open/user/notification/init',
].includes(req.path)
) {
return next();
}
const userService = Container.get(UserService);
const authInfo = await userService.getUserInfo();
const authInfo =
(await shareStore.getAuthInfo()) || ({} as IKeyvStore['authInfo']);
let isInitialized = true;
if (
@@ -125,10 +152,15 @@ export default ({ app }: { app: Application }) => {
app.use(rewrite('/open/*', '/api/$1'));
app.use(config.api.prefix, routes());
app.use((req, res, next) => {
const err: any = new Error('Not Found');
err['status'] = 404;
next(err);
app.get('*', (_, res, next) => {
const indexPath = path.join(frontendPath, 'index.html');
res.sendFile(indexPath, (err) => {
if (err) {
const err: any = new Error('Not Found');
err['status'] = 404;
next(err);
}
});
});
app.use(errors());
@@ -162,8 +194,8 @@ export default ({ app }: { app: Application }) => {
.status(500)
.send({
code: 400,
message: `${err.name} ${err.message}`,
validation: err.errors,
message: `${err.message}`,
errors: err.errors,
})
.end();
}
@@ -178,8 +210,6 @@ export default ({ app }: { app: Application }) => {
res: Response,
next: NextFunction,
) => {
Sentry.captureException(err);
res.status(err.status || 500);
res.json({
code: err.status || 500,
+76 -17
View File
@@ -11,35 +11,90 @@ import { CrontabViewModel, CronViewType } from '../data/cronView';
import { initPosition } from '../data/env';
import { AuthDataType, SystemModel } from '../data/system';
import SystemService from '../services/system';
import UserService from '../services/user';
import { writeFile, readFile } from 'fs/promises';
import { createRandomString, fileExist, isDemoEnv, safeJSONParse } from '../config/util';
import OpenService from '../services/open';
import { shareStore } from '../shared/store';
import Logger from './logger';
import { AppModel } from '../data/open';
export default async () => {
const cronService = Container.get(CronService);
const envService = Container.get(EnvService);
const dependenceService = Container.get(DependenceService);
const systemService = Container.get(SystemService);
const userService = Container.get(UserService);
const openService = Container.get(OpenService);
const installDependencies = () => {
// 初始化时安装所有处于安装中,安装成功,安装失败的依赖
DependenceModel.findAll({
// 初始化增加系统配置
let systemApp = (
await AppModel.findOne({
where: { name: 'system' },
})
)?.get({ plain: true });
if (!systemApp) {
systemApp = await AppModel.create({
name: 'system',
scopes: ['crons', 'system'],
client_id: createRandomString(12, 12),
client_secret: createRandomString(24, 24),
});
}
const [systemConfig] = await SystemModel.findOrCreate({
where: { type: AuthDataType.systemConfig },
});
await SystemModel.findOrCreate({
where: { type: AuthDataType.notification },
});
const [authConfig] = await SystemModel.findOrCreate({
where: { type: AuthDataType.authConfig },
});
if (!authConfig?.info || isDemoEnv()) {
let authInfo = {
username: 'admin',
password: 'admin',
};
try {
const authFileExist = await fileExist(config.authConfigFile);
if (authFileExist) {
const content = await readFile(config.authConfigFile, 'utf8');
authInfo = safeJSONParse(content);
}
} catch (error) {
Logger.warn('Failed to read auth config file, using default credentials');
}
await SystemModel.upsert({
id: authConfig?.id,
info: authInfo,
type: AuthDataType.authConfig,
});
}
const installDependencies = async () => {
const docs = await DependenceModel.findAll({
where: {},
order: [
['type', 'DESC'],
['createdAt', 'DESC'],
],
raw: true,
}).then(async (docs) => {
await DependenceModel.update(
{ status: DependenceStatus.queued, log: [] },
{ where: { id: docs.map((x) => x.id!) } },
);
setTimeout(() => {
dependenceService.installDependenceOneByOne(docs);
}, 5000);
});
await DependenceModel.update(
{ status: DependenceStatus.queued, log: [] },
{ where: { id: docs.map((x) => x.id!) } },
);
setTimeout(async () => {
await dependenceService.installDependenceOneByOne(docs);
const bootAfterLoader = await import('./bootAfter');
bootAfterLoader.default();
}, 5000);
};
// 初始化更新 linux/python/nodejs 镜像源配置
const systemConfig = await systemService.getSystemConfig();
if (systemConfig.info?.pythonMirror) {
systemService.updatePythonMirror({
pythonMirror: systemConfig.info?.pythonMirror,
@@ -121,7 +176,7 @@ export default async () => {
if (doc.command.includes(`${config.rootPath}/log/`)) {
await CrontabModel.update(
{
command: `${config.rootPath}/data/log/${doc.command.replace(
command: `${config.dataPath}/log/${doc.command.replace(
`${config.rootPath}/log/`,
'',
)}`,
@@ -132,7 +187,7 @@ export default async () => {
if (doc.command.includes(`${config.rootPath}/config/`)) {
await CrontabModel.update(
{
command: `${config.rootPath}/data/config/${doc.command.replace(
command: `${config.dataPath}/config/${doc.command.replace(
`${config.rootPath}/config/`,
'',
)}`,
@@ -143,7 +198,7 @@ export default async () => {
if (doc.command.includes(`${config.rootPath}/db/`)) {
await CrontabModel.update(
{
command: `${config.rootPath}/data/db/${doc.command.replace(
command: `${config.dataPath}/db/${doc.command.replace(
`${config.rootPath}/db/`,
'',
)}`,
@@ -159,6 +214,10 @@ export default async () => {
await cronService.autosave_crontab();
await envService.set_envs();
// 初始化增加系统配置
await SystemModel.upsert({ type: AuthDataType.systemConfig });
const authInfo = await userService.getAuthInfo();
const apps = await openService.findApps();
await shareStore.updateAuthInfo(authInfo);
if (apps?.length) {
await shareStore.updateApps(apps);
}
};
+96 -83
View File
@@ -3,9 +3,16 @@ import path from 'path';
import os from 'os';
import Logger from './logger';
import { fileExist } from '../config/util';
import { writeFileWithLock } from '../shared/utils';
const rootPath = process.env.QL_DIR as string;
const dataPath = path.join(rootPath, 'data/');
let dataPath = path.join(rootPath, 'data/');
if (process.env.QL_DATA_DIR) {
dataPath = process.env.QL_DATA_DIR.replace(/\/$/g, '');
}
const preloadPath = path.join(rootPath, 'shell/preload/');
const configPath = path.join(dataPath, 'config/');
const scriptPath = path.join(dataPath, 'scripts/');
const logPath = path.join(dataPath, 'log/');
@@ -14,105 +21,111 @@ const bakPath = path.join(dataPath, 'bak/');
const samplePath = path.join(rootPath, 'sample/');
const tmpPath = path.join(logPath, '.tmp/');
const confFile = path.join(configPath, 'config.sh');
const authConfigFile = path.join(configPath, 'auth.json');
const sampleConfigFile = path.join(samplePath, 'config.sample.sh');
const sampleAuthFile = path.join(samplePath, 'auth.sample.json');
const sampleTaskShellFile = path.join(samplePath, 'task.sample.sh');
const sampleNotifyJsFile = path.join(samplePath, 'notify.js');
const sampleNotifyPyFile = path.join(samplePath, 'notify.py');
const scriptNotifyJsFile = path.join(scriptPath, 'sendNotify.js');
const scriptNotifyPyFile = path.join(scriptPath, 'notify.py');
const jsNotifyFile = path.join(preloadPath, '__ql_notify__.js');
const pyNotifyFile = path.join(preloadPath, '__ql_notify__.py');
const TaskBeforeFile = path.join(configPath, 'task_before.sh');
const TaskBeforeJsFile = path.join(configPath, 'task_before.js');
const TaskBeforePyFile = path.join(configPath, 'task_before.py');
const TaskAfterFile = path.join(configPath, 'task_after.sh');
const homedir = os.homedir();
const sshPath = path.resolve(homedir, '.ssh');
const sshdPath = path.join(dataPath, 'ssh.d');
const systemLogPath = path.join(dataPath, 'syslog');
const directories = [
configPath,
scriptPath,
preloadPath,
logPath,
tmpPath,
uploadPath,
sshPath,
bakPath,
sshdPath,
systemLogPath,
];
const files = [
{
target: confFile,
source: sampleConfigFile,
checkExistence: true,
},
{
target: jsNotifyFile,
source: sampleNotifyJsFile,
checkExistence: false,
},
{
target: pyNotifyFile,
source: sampleNotifyPyFile,
checkExistence: false,
},
{
target: scriptNotifyJsFile,
source: sampleNotifyJsFile,
checkExistence: true,
},
{
target: scriptNotifyPyFile,
source: sampleNotifyPyFile,
checkExistence: true,
},
{
target: TaskBeforeFile,
source: sampleTaskShellFile,
checkExistence: true,
},
{
target: TaskBeforeJsFile,
content:
'// The JavaScript code that executes before the JavaScript task execution will execute.',
checkExistence: true,
},
{
target: TaskBeforePyFile,
content:
'# The Python code that executes before the Python task execution will execute.',
checkExistence: true,
},
{
target: TaskAfterFile,
source: sampleTaskShellFile,
checkExistence: true,
},
];
export default async () => {
const authFileExist = await fileExist(authConfigFile);
const confFileExist = await fileExist(confFile);
const scriptDirExist = await fileExist(scriptPath);
const logDirExist = await fileExist(logPath);
const configDirExist = await fileExist(configPath);
const uploadDirExist = await fileExist(uploadPath);
const sshDirExist = await fileExist(sshPath);
const bakDirExist = await fileExist(bakPath);
const sshdDirExist = await fileExist(sshdPath);
const systemLogDirExist = await fileExist(systemLogPath);
const tmpDirExist = await fileExist(tmpPath);
const scriptNotifyJsFileExist = await fileExist(scriptNotifyJsFile);
const scriptNotifyPyFileExist = await fileExist(scriptNotifyPyFile);
const TaskBeforeFileExist = await fileExist(TaskBeforeFile);
const TaskAfterFileExist = await fileExist(TaskAfterFile);
if (!configDirExist) {
await fs.mkdir(configPath);
for (const dirPath of directories) {
if (!(await fileExist(dirPath))) {
try {
await fs.mkdir(dirPath, { recursive: true });
} catch (err: any) {
Logger.warn(`Unable to create directory ${dirPath}: ${err.message}`);
}
}
}
if (!scriptDirExist) {
await fs.mkdir(scriptPath);
}
if (!logDirExist) {
await fs.mkdir(logPath);
}
if (!tmpDirExist) {
await fs.mkdir(tmpPath);
}
if (!uploadDirExist) {
await fs.mkdir(uploadPath);
}
if (!sshDirExist) {
await fs.mkdir(sshPath);
}
if (!bakDirExist) {
await fs.mkdir(bakPath);
}
if (!sshdDirExist) {
await fs.mkdir(sshdPath);
}
if (!systemLogDirExist) {
await fs.mkdir(systemLogPath);
}
// 初始化文件
if (!authFileExist) {
await fs.writeFile(authConfigFile, await fs.readFile(sampleAuthFile));
}
if (!confFileExist) {
await fs.writeFile(confFile, await fs.readFile(sampleConfigFile));
}
if (!scriptNotifyJsFileExist) {
await fs.writeFile(
scriptNotifyJsFile,
await fs.readFile(sampleNotifyJsFile),
);
}
if (!scriptNotifyPyFileExist) {
await fs.writeFile(
scriptNotifyPyFile,
await fs.readFile(sampleNotifyPyFile),
);
}
if (!TaskBeforeFileExist) {
await fs.writeFile(TaskBeforeFile, await fs.readFile(sampleTaskShellFile));
}
if (!TaskAfterFileExist) {
await fs.writeFile(TaskAfterFile, await fs.readFile(sampleTaskShellFile));
for (const item of files) {
const exists = await fileExist(item.target);
if (!item.checkExistence || !exists) {
if (!item.content && !item.source) {
throw new Error(
`Neither content nor source specified for ${item.target}`,
);
}
const content =
item.content ||
(await fs.readFile(item.source!, { encoding: 'utf-8' }));
await writeFileWithLock(item.target, content);
}
}
Logger.info('✌️ Init file down');
console.log('✌️ Init file down');
};
+39 -16
View File
@@ -2,6 +2,7 @@ import { Container } from 'typedi';
import SystemService from '../services/system';
import ScheduleService, { ScheduleTaskType } from '../services/schedule';
import SubscriptionService from '../services/subscription';
import SshKeyService from '../services/sshKey';
import config from '../config';
import { fileExist } from '../config/util';
import { join } from 'path';
@@ -10,9 +11,13 @@ export default async () => {
const systemService = Container.get(SystemService);
const scheduleService = Container.get(ScheduleService);
const subscriptionService = Container.get(SubscriptionService);
const sshKeyService = Container.get(SshKeyService);
// 生成内置token
let tokenCommand = `tsx ${join(config.rootPath, 'back/token.ts')}`;
let tokenCommand = `ts-node-transpile-only ${join(
config.rootPath,
'back/token.ts',
)}`;
const tokenFile = join(config.rootPath, 'static/build/token.js');
if (await fileExist(tokenFile)) {
@@ -22,30 +27,48 @@ export default async () => {
id: NaN,
name: '生成token',
command: tokenCommand,
runOrigin: 'system',
} as ScheduleTaskType;
await scheduleService.cancelIntervalTask(cron);
scheduleService.createIntervalTask(cron, {
days: 28,
});
scheduleService.createIntervalTask(
cron,
{
days: 28,
},
true,
);
// 运行删除日志任务
const data = await systemService.getSystemConfig();
if (data && data.info && data.info.logRemoveFrequency) {
const rmlogCron = {
id: data.id as number,
name: '删除日志',
command: `ql rmlog ${data.info.logRemoveFrequency}`,
};
await scheduleService.cancelIntervalTask(rmlogCron);
scheduleService.createIntervalTask(rmlogCron, {
days: data.info.logRemoveFrequency,
});
if (data && data.info) {
if (data.info.logRemoveFrequency) {
const rmlogCron = {
id: data.id as number,
name: '删除日志',
command: `ql rmlog ${data.info.logRemoveFrequency}`,
runOrigin: 'system' as const,
};
await scheduleService.cancelIntervalTask(rmlogCron);
scheduleService.createIntervalTask(
rmlogCron,
{
days: data.info.logRemoveFrequency,
},
true,
);
}
systemService.updateTimezone(data.info);
// Apply global SSH key if configured
if (data.info.globalSshKey) {
await sshKeyService.addGlobalSSHKey(data.info.globalSshKey, 'global');
}
}
// 运行所有订阅
await subscriptionService.setSshConfig();
const subs = await subscriptionService.list();
for (const sub of subs) {
subscriptionService.handleTask(sub, !sub.is_disabled, !sub.is_disabled);
subscriptionService.handleTask(sub.get({ plain: true }), !sub.is_disabled);
}
};
+50 -25
View File
@@ -4,35 +4,60 @@ import config from '../config';
import path from 'path';
const levelMap: Record<string, string> = {
info: '🔵',
warn: '🟡',
error: '🔴',
debug: '🔶'
}
const customFormat = winston.format.combine(
winston.format.splat(),
winston.format.timestamp({ format: "YYYY-MM-DD HH:mm:ss" }),
winston.format.align(),
winston.format.printf((i) => `[${levelMap[i.level]}${i.level}] [${[i.timestamp]}]: ${i.message}`),
);
const defaultOptions = {
format: customFormat,
datePattern: "YYYY-MM-DD",
maxSize: "20m",
maxFiles: "7d",
info: '', // info图标
warn: '⚠️', // 警告图标
error: '', // 错误图标
debug: '🐛', // debug调试图标
};
const baseFormat = [
winston.format.splat(),
winston.format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss' }),
winston.format.align(),
];
const consoleFormat = winston.format.combine(
winston.format.colorize({ level: true }),
...baseFormat,
winston.format.printf((info) => {
return `[${info.level} ${info.timestamp}]:${info.message}`;
}),
);
const plainFormat = winston.format.combine(
winston.format.uncolorize(),
...baseFormat,
winston.format.printf((info) => {
return `[${levelMap[info.level] || ''}${info.level} ${info.timestamp}]:${
info.message
}`;
}),
);
const consoleTransport = new winston.transports.Console({
format: consoleFormat,
level: 'debug',
});
const fileTransport = new winston.transports.DailyRotateFile({
filename: path.join(config.systemLogPath, '%DATE%.log'),
datePattern: 'YYYY-MM-DD',
maxSize: '20m',
maxFiles: '7d',
format: plainFormat,
level: config.logs.level || 'info',
});
const LoggerInstance = winston.createLogger({
level: config.logs.level,
level: 'debug',
levels: winston.config.npm.levels,
transports: [
new winston.transports.DailyRotateFile({
filename: path.join(config.systemLogPath, '%DATE%.log'),
...defaultOptions,
})
],
transports: [consoleTransport, fileTransport],
exceptionHandlers: [consoleTransport, fileTransport],
rejectionHandlers: [consoleTransport, fileTransport],
});
LoggerInstance.on('error', (error) => {
console.error('Logger error:', error);
});
export default LoggerInstance;
-32
View File
@@ -1,32 +0,0 @@
import { Application } from 'express';
import * as Sentry from '@sentry/node';
import Logger from './logger';
import config from '../config';
import fs from 'fs';
import { parseVersion } from '../config/util';
export default async ({ expressApp }: { expressApp: Application }) => {
const { version } = await parseVersion(config.versionFile);
Sentry.init({
ignoreErrors: [
/SequelizeUniqueConstraintError/i,
/Validation error/i,
/UnauthorizedError/i,
/celebrate request validation failed/i,
],
dsn: 'https://8b5c84cfef3e22541bc84de0ed00497b@o1098464.ingest.sentry.io/6122819',
integrations: [
new Sentry.Integrations.Http({ tracing: true }),
new Sentry.Integrations.Express({ app: expressApp }),
],
tracesSampleRate: 0.8,
release: version,
});
expressApp.use(Sentry.Handlers.requestHandler());
expressApp.use(Sentry.Handlers.tracingHandler());
Logger.info('✌️ Sentry loaded');
console.log('✌️ Sentry loaded');
};
-22
View File
@@ -5,28 +5,6 @@ import Sock from './sock';
export default async ({ server }: { server: Server }) => {
await Sock({ server });
Logger.info('✌️ Sock loaded');
let exitTime = 0;
let timer: NodeJS.Timeout;
process.on('SIGINT', (singal) => {
Logger.warn(`Server need close, singal ${singal}`);
console.warn(`Server need close, singal ${singal}`);
exitTime++;
if (exitTime >= 3) {
Logger.warn('Forcing server close');
console.warn('Forcing server close');
clearTimeout(timer);
process.exit(1);
}
server.close(() => {
if (timer) {
clearTimeout(timer);
}
timer = setTimeout(() => {
process.exit();
}, 15000);
});
});
process.on('uncaughtException', (error) => {
Logger.error('Uncaught exception:', error);
+16 -20
View File
@@ -2,12 +2,13 @@ import sockJs from 'sockjs';
import { Server } from 'http';
import { Container } from 'typedi';
import SockService from '../services/sock';
import config from '../config/index';
import fs from 'fs/promises';
import { getPlatform, safeJSONParse } from '../config/util';
import { getPlatform } from '../config/util';
import { shareStore } from '../shared/store';
import { isValidToken } from '../shared/auth';
import config from '../config';
export default async ({ server }: { server: Server }) => {
const echo = sockJs.createServer({ prefix: '/api/ws', log: () => {} });
const echo = sockJs.createServer({ prefix: `${config.baseUrl}/api/ws`, log: () => { } });
const sockService = Container.get(SockService);
echo.on('connection', async (conn) => {
@@ -15,27 +16,22 @@ export default async ({ server }: { server: Server }) => {
conn.close('404');
}
const data = await fs.readFile(config.authConfigFile, 'utf8');
const authInfo = await shareStore.getAuthInfo();
const platform = getPlatform(conn.headers['user-agent'] || '') || 'desktop';
const headerToken = conn.url.replace(`${conn.pathname}?token=`, '');
if (data) {
const { token = '', tokens = {} } = safeJSONParse(data);
if (headerToken === token || tokens[platform] === headerToken) {
conn.write(JSON.stringify({ type: 'ping', message: 'hanhh' }));
sockService.addClient(conn);
conn.on('data', (message) => {
conn.write(message);
});
if (isValidToken(authInfo, headerToken, platform)) {
sockService.addClient(conn);
conn.on('close', function () {
sockService.removeClient(conn);
});
conn.on('data', (message) => {
conn.write(message);
});
return;
} else {
conn.write(JSON.stringify({ type: 'ping', message: 'whyour' }));
}
conn.on('close', function () {
sockService.removeClient(conn);
});
return;
}
conn.close('404');
-93
View File
@@ -1,93 +0,0 @@
import bodyParser from 'body-parser';
import { errors } from 'celebrate';
import cors from 'cors';
import { Application, NextFunction, Request, Response } from 'express';
import jwt from 'express-jwt';
import Container from 'typedi';
import { Logger } from 'winston';
import config from '../config';
import SystemService from '../services/system';
export default ({ app }: { app: Application }) => {
app.set('trust proxy', 'loopback');
app.use(cors());
app.use(bodyParser.json({ limit: '50mb' }));
app.use(bodyParser.urlencoded({ limit: '50mb', extended: true }));
app.use(
jwt({
secret: config.secret,
algorithms: ['HS384'],
}),
);
app.put(
'/api/system',
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('system');
res.send(result);
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
},
);
app.put(
'/api/data',
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('data');
res.send(result);
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
},
);
app.use((req, res, next) => {
const err: any = new Error('Not Found');
err['status'] = 404;
next(err);
});
app.use(errors());
app.use(
(
err: Error & { status: number },
req: Request,
res: Response,
next: NextFunction,
) => {
if (err.name === 'UnauthorizedError') {
return res
.status(err.status)
.send({ code: 401, message: err.message })
.end();
}
return next(err);
},
);
app.use(
(
err: Error & { status: number },
req: Request,
res: Response,
next: NextFunction,
) => {
res.status(err.status || 500);
res.json({
code: err.status || 500,
message: err.message,
});
},
);
};
+80
View File
@@ -0,0 +1,80 @@
import { Request, Response, NextFunction } from 'express';
import Logger from '../loaders/logger';
import { performance } from 'perf_hooks';
import { metricsService } from '../services/metrics';
interface RequestMetrics {
method: string;
path: string;
duration: number;
statusCode: number;
timestamp: number;
platform?: string;
}
const requestMetrics: RequestMetrics[] = [];
export const monitoringMiddleware = (
req: Request,
res: Response,
next: NextFunction,
) => {
const start = performance.now();
const originalEnd = res.end;
res.end = function (chunk?: any, encoding?: any, cb?: any) {
const duration = performance.now() - start;
const metric: RequestMetrics = {
method: req.method,
path: req.path,
duration,
statusCode: res.statusCode,
timestamp: Date.now(),
platform: req.platform,
};
requestMetrics.push(metric);
metricsService.record('http_request', duration, {
method: req.method,
path: req.path,
statusCode: res.statusCode.toString(),
...(req.platform && { platform: req.platform }),
});
if (requestMetrics.length > 1000) {
requestMetrics.shift();
}
if (duration > 1000) {
Logger.warn(
`Slow request detected: ${req.method} ${
req.path
} took ${duration.toFixed(2)}ms`,
);
}
return originalEnd.call(this, chunk, encoding, cb);
};
next();
};
export const getMetrics = () => {
return {
totalRequests: requestMetrics.length,
averageDuration:
requestMetrics.reduce((acc, curr) => acc + curr.duration, 0) /
requestMetrics.length,
requestsByMethod: requestMetrics.reduce((acc, curr) => {
acc[curr.method] = (acc[curr.method] || 0) + 1;
return acc;
}, {} as Record<string, number>),
requestsByPlatform: requestMetrics.reduce((acc, curr) => {
if (curr.platform) {
acc[curr.platform] = (acc[curr.platform] || 0) + 1;
}
return acc;
}, {} as Record<string, number>),
recentRequests: requestMetrics.slice(-10),
};
};
+275
View File
@@ -0,0 +1,275 @@
syntax = "proto3";
package com.ql.api;
message EnvItem {
optional int32 id = 1;
optional string name = 2;
optional string value = 3;
optional string remarks = 4;
optional int32 status = 5;
optional int64 position = 6;
}
message GetEnvsRequest { string searchValue = 1; }
message CreateEnvRequest { repeated EnvItem envs = 1; }
message UpdateEnvRequest { EnvItem env = 1; }
message DeleteEnvsRequest { repeated int32 ids = 1; }
message MoveEnvRequest {
int32 id = 1;
int32 fromIndex = 2;
int32 toIndex = 3;
}
message DisableEnvsRequest { repeated int32 ids = 1; }
message EnableEnvsRequest { repeated int32 ids = 1; }
message UpdateEnvNamesRequest {
repeated int32 ids = 1;
string name = 2;
}
message GetEnvByIdRequest { int32 id = 1; }
message EnvsResponse {
int32 code = 1;
repeated EnvItem data = 2;
optional string message = 3;
}
message EnvResponse {
int32 code = 1;
EnvItem data = 2;
optional string message = 3;
}
message Response {
int32 code = 1;
optional string message = 2;
}
message ExtraScheduleItem { string schedule = 1; }
message CronItem {
optional int32 id = 1;
optional string command = 2;
optional string schedule = 3;
optional string name = 4;
repeated string labels = 5;
optional int32 sub_id = 6;
repeated ExtraScheduleItem extra_schedules = 7;
optional string task_before = 8;
optional string task_after = 9;
optional int32 status = 10;
optional string log_path = 11;
optional int32 pid = 12;
optional int64 last_running_time = 13;
optional int64 last_execution_time = 14;
}
message CreateCronRequest {
string command = 1;
string schedule = 2;
optional string name = 3;
repeated string labels = 4;
optional int32 sub_id = 5;
repeated ExtraScheduleItem extra_schedules = 6;
optional string task_before = 7;
optional string task_after = 8;
}
message UpdateCronRequest {
int32 id = 1;
optional string command = 2;
optional string schedule = 3;
optional string name = 4;
repeated string labels = 5;
optional int32 sub_id = 6;
repeated ExtraScheduleItem extra_schedules = 7;
optional string task_before = 8;
optional string task_after = 9;
}
message DeleteCronsRequest { repeated int32 ids = 1; }
message GetCronsRequest {
optional string searchValue = 1;
}
message GetCronByIdRequest { int32 id = 1; }
message EnableCronsRequest { repeated int32 ids = 1; }
message DisableCronsRequest { repeated int32 ids = 1; }
message RunCronsRequest { repeated int32 ids = 1; }
message CronsResponse {
int32 code = 1;
repeated CronItem data = 2;
optional string message = 3;
}
message CronResponse {
int32 code = 1;
CronItem data = 2;
optional string message = 3;
}
message CronDetailRequest { string log_path = 1; }
message CronDetailResponse {
int32 code = 1;
CronItem data = 2;
optional string message = 3;
}
enum NotificationMode {
gotify = 0;
goCqHttpBot = 1;
serverChan = 2;
pushDeer = 3;
bark = 4;
chat = 5;
telegramBot = 6;
dingtalkBot = 7;
weWorkBot = 8;
weWorkApp = 9;
aibotk = 10;
iGot = 11;
pushPlus = 12;
wePlusBot = 13;
email = 14;
pushMe = 15;
feishu = 16;
webhook = 17;
chronocat = 18;
ntfy = 19;
wxPusherBot = 20;
}
message NotificationInfo {
NotificationMode type = 1;
optional string gotifyUrl = 2;
optional string gotifyToken = 3;
optional int32 gotifyPriority = 4;
optional string goCqHttpBotUrl = 5;
optional string goCqHttpBotToken = 6;
optional string goCqHttpBotQq = 7;
optional string serverChanKey = 8;
optional string pushDeerKey = 9;
optional string pushDeerUrl = 10;
optional string synologyChatUrl = 11;
optional string barkPush = 12;
optional string barkIcon = 13;
optional string barkSound = 14;
optional string barkGroup = 15;
optional string barkLevel = 16;
optional string barkUrl = 17;
optional string barkArchive = 18;
optional string telegramBotToken = 19;
optional string telegramBotUserId = 20;
optional string telegramBotProxyHost = 21;
optional string telegramBotProxyPort = 22;
optional string telegramBotProxyAuth = 23;
optional string telegramBotApiHost = 24;
optional string dingtalkBotToken = 25;
optional string dingtalkBotSecret = 26;
optional string weWorkBotKey = 27;
optional string weWorkOrigin = 28;
optional string weWorkAppKey = 29;
optional string aibotkKey = 30;
optional string aibotkType = 31;
optional string aibotkName = 32;
optional string iGotPushKey = 33;
optional string pushPlusToken = 34;
optional string pushPlusUser = 35;
optional string pushPlusTemplate = 36;
optional string pushplusChannel = 37;
optional string pushplusWebhook = 38;
optional string pushplusCallbackUrl = 39;
optional string pushplusTo = 40;
optional string wePlusBotToken = 41;
optional string wePlusBotReceiver = 42;
optional string wePlusBotVersion = 43;
optional string emailService = 44;
optional string emailUser = 45;
optional string emailPass = 46;
optional string emailTo = 47;
optional string pushMeKey = 48;
optional string pushMeUrl = 49;
optional string chronocatURL = 50;
optional string chronocatQQ = 51;
optional string chronocatToken = 52;
optional string webhookHeaders = 53;
optional string webhookBody = 54;
optional string webhookUrl = 55;
optional string webhookMethod = 56;
optional string webhookContentType = 57;
optional string larkKey = 58;
optional string larkSecret = 69;
optional string ntfyUrl = 59;
optional string ntfyTopic = 60;
optional string ntfyPriority = 61;
optional string ntfyToken = 62;
optional string ntfyUsername = 63;
optional string ntfyPassword = 64;
optional string ntfyActions = 65;
optional string wxPusherBotAppToken = 66;
optional string wxPusherBotTopicIds = 67;
optional string wxPusherBotUids = 68;
}
message SystemNotifyRequest {
string title = 1;
string content = 2;
optional NotificationInfo notificationInfo = 3;
}
service Api {
rpc GetEnvs(GetEnvsRequest) returns (EnvsResponse) {}
rpc CreateEnv(CreateEnvRequest) returns (EnvsResponse) {}
rpc UpdateEnv(UpdateEnvRequest) returns (EnvResponse) {}
rpc DeleteEnvs(DeleteEnvsRequest) returns (Response) {}
rpc MoveEnv(MoveEnvRequest) returns (EnvResponse) {}
rpc DisableEnvs(DisableEnvsRequest) returns (Response) {}
rpc EnableEnvs(EnableEnvsRequest) returns (Response) {}
rpc UpdateEnvNames(UpdateEnvNamesRequest) returns (Response) {}
rpc GetEnvById(GetEnvByIdRequest) returns (EnvResponse) {}
rpc SystemNotify(SystemNotifyRequest) returns (Response) {}
rpc GetCronDetail(CronDetailRequest) returns (CronDetailResponse) {}
rpc CreateCron(CreateCronRequest) returns (CronResponse) {}
rpc UpdateCron(UpdateCronRequest) returns (CronResponse) {}
rpc DeleteCrons(DeleteCronsRequest) returns (Response) {}
rpc GetCrons(GetCronsRequest) returns (CronsResponse) {}
rpc GetCronById(GetCronByIdRequest) returns (CronResponse) {}
rpc EnableCrons(EnableCronsRequest) returns (Response) {}
rpc DisableCrons(DisableCronsRequest) returns (Response) {}
rpc RunCrons(RunCronsRequest) returns (Response) {}
}
+4740
View File
File diff suppressed because it is too large Load Diff
+114 -91
View File
@@ -1,17 +1,23 @@
// Code generated by protoc-gen-ts_proto. DO NOT EDIT.
// versions:
// protoc-gen-ts_proto v2.6.1
// protoc v3.21.12
// source: back/protos/cron.proto
/* eslint-disable */
import { BinaryReader, BinaryWriter } from "@bufbuild/protobuf/wire";
import {
CallOptions,
type CallOptions,
ChannelCredentials,
Client,
ClientOptions,
ClientUnaryCall,
handleUnaryCall,
type ClientOptions,
type ClientUnaryCall,
type handleUnaryCall,
makeGenericClientConstructor,
Metadata,
ServiceError,
UntypedServiceImplementation,
type ServiceError,
type UntypedServiceImplementation,
} from "@grpc/grpc-js";
import _m0 from "protobufjs/minimal";
export const protobufPackage = "com.ql.cron";
@@ -23,7 +29,7 @@ export interface ICron {
id: string;
schedule: string;
command: string;
extraSchedules: ISchedule[];
extra_schedules: ISchedule[];
name: string;
}
@@ -45,51 +51,53 @@ function createBaseISchedule(): ISchedule {
return { schedule: "" };
}
export const ISchedule = {
encode(message: ISchedule, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const ISchedule: MessageFns<ISchedule> = {
encode(message: ISchedule, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.schedule !== "") {
writer.uint32(10).string(message.schedule);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): ISchedule {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): ISchedule {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseISchedule();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.schedule = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): ISchedule {
return { schedule: isSet(object.schedule) ? String(object.schedule) : "" };
return { schedule: isSet(object.schedule) ? globalThis.String(object.schedule) : "" };
},
toJSON(message: ISchedule): unknown {
const obj: any = {};
message.schedule !== undefined && (obj.schedule = message.schedule);
if (message.schedule !== "") {
obj.schedule = message.schedule;
}
return obj;
},
create<I extends Exact<DeepPartial<ISchedule>, I>>(base?: I): ISchedule {
return ISchedule.fromPartial(base ?? {});
return ISchedule.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<ISchedule>, I>>(object: I): ISchedule {
const message = createBaseISchedule();
message.schedule = object.schedule ?? "";
@@ -98,11 +106,11 @@ export const ISchedule = {
};
function createBaseICron(): ICron {
return { id: "", schedule: "", command: "", extraSchedules: [], name: "" };
return { id: "", schedule: "", command: "", extra_schedules: [], name: "" };
}
export const ICron = {
encode(message: ICron, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const ICron: MessageFns<ICron> = {
encode(message: ICron, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.id !== "") {
writer.uint32(10).string(message.id);
}
@@ -112,8 +120,8 @@ export const ICron = {
if (message.command !== "") {
writer.uint32(26).string(message.command);
}
for (const v of message.extraSchedules) {
ISchedule.encode(v!, writer.uint32(34).fork()).ldelim();
for (const v of message.extra_schedules) {
ISchedule.encode(v!, writer.uint32(34).fork()).join();
}
if (message.name !== "") {
writer.uint32(42).string(message.name);
@@ -121,93 +129,103 @@ export const ICron = {
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): ICron {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): ICron {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseICron();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.id = reader.string();
continue;
case 2:
}
case 2: {
if (tag !== 18) {
break;
}
message.schedule = reader.string();
continue;
case 3:
}
case 3: {
if (tag !== 26) {
break;
}
message.command = reader.string();
continue;
case 4:
}
case 4: {
if (tag !== 34) {
break;
}
message.extraSchedules.push(ISchedule.decode(reader, reader.uint32()));
message.extra_schedules.push(ISchedule.decode(reader, reader.uint32()));
continue;
case 5:
}
case 5: {
if (tag !== 42) {
break;
}
message.name = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): ICron {
return {
id: isSet(object.id) ? String(object.id) : "",
schedule: isSet(object.schedule) ? String(object.schedule) : "",
command: isSet(object.command) ? String(object.command) : "",
extraSchedules: Array.isArray(object?.extraSchedules)
? object.extraSchedules.map((e: any) => ISchedule.fromJSON(e))
id: isSet(object.id) ? globalThis.String(object.id) : "",
schedule: isSet(object.schedule) ? globalThis.String(object.schedule) : "",
command: isSet(object.command) ? globalThis.String(object.command) : "",
extra_schedules: globalThis.Array.isArray(object?.extra_schedules)
? object.extra_schedules.map((e: any) => ISchedule.fromJSON(e))
: [],
name: isSet(object.name) ? String(object.name) : "",
name: isSet(object.name) ? globalThis.String(object.name) : "",
};
},
toJSON(message: ICron): unknown {
const obj: any = {};
message.id !== undefined && (obj.id = message.id);
message.schedule !== undefined && (obj.schedule = message.schedule);
message.command !== undefined && (obj.command = message.command);
if (message.extraSchedules) {
obj.extraSchedules = message.extraSchedules.map((e) => e ? ISchedule.toJSON(e) : undefined);
} else {
obj.extraSchedules = [];
if (message.id !== "") {
obj.id = message.id;
}
if (message.schedule !== "") {
obj.schedule = message.schedule;
}
if (message.command !== "") {
obj.command = message.command;
}
if (message.extra_schedules?.length) {
obj.extra_schedules = message.extra_schedules.map((e) => ISchedule.toJSON(e));
}
if (message.name !== "") {
obj.name = message.name;
}
message.name !== undefined && (obj.name = message.name);
return obj;
},
create<I extends Exact<DeepPartial<ICron>, I>>(base?: I): ICron {
return ICron.fromPartial(base ?? {});
return ICron.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<ICron>, I>>(object: I): ICron {
const message = createBaseICron();
message.id = object.id ?? "";
message.schedule = object.schedule ?? "";
message.command = object.command ?? "";
message.extraSchedules = object.extraSchedules?.map((e) => ISchedule.fromPartial(e)) || [];
message.extra_schedules = object.extra_schedules?.map((e) => ISchedule.fromPartial(e)) || [];
message.name = object.name ?? "";
return message;
},
@@ -217,55 +235,53 @@ function createBaseAddCronRequest(): AddCronRequest {
return { crons: [] };
}
export const AddCronRequest = {
encode(message: AddCronRequest, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const AddCronRequest: MessageFns<AddCronRequest> = {
encode(message: AddCronRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
for (const v of message.crons) {
ICron.encode(v!, writer.uint32(10).fork()).ldelim();
ICron.encode(v!, writer.uint32(10).fork()).join();
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): AddCronRequest {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): AddCronRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseAddCronRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.crons.push(ICron.decode(reader, reader.uint32()));
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): AddCronRequest {
return { crons: Array.isArray(object?.crons) ? object.crons.map((e: any) => ICron.fromJSON(e)) : [] };
return { crons: globalThis.Array.isArray(object?.crons) ? object.crons.map((e: any) => ICron.fromJSON(e)) : [] };
},
toJSON(message: AddCronRequest): unknown {
const obj: any = {};
if (message.crons) {
obj.crons = message.crons.map((e) => e ? ICron.toJSON(e) : undefined);
} else {
obj.crons = [];
if (message.crons?.length) {
obj.crons = message.crons.map((e) => ICron.toJSON(e));
}
return obj;
},
create<I extends Exact<DeepPartial<AddCronRequest>, I>>(base?: I): AddCronRequest {
return AddCronRequest.fromPartial(base ?? {});
return AddCronRequest.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<AddCronRequest>, I>>(object: I): AddCronRequest {
const message = createBaseAddCronRequest();
message.crons = object.crons?.map((e) => ICron.fromPartial(e)) || [];
@@ -277,13 +293,13 @@ function createBaseAddCronResponse(): AddCronResponse {
return {};
}
export const AddCronResponse = {
encode(_: AddCronResponse, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const AddCronResponse: MessageFns<AddCronResponse> = {
encode(_: AddCronResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): AddCronResponse {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): AddCronResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseAddCronResponse();
while (reader.pos < end) {
@@ -293,7 +309,7 @@ export const AddCronResponse = {
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -308,9 +324,8 @@ export const AddCronResponse = {
},
create<I extends Exact<DeepPartial<AddCronResponse>, I>>(base?: I): AddCronResponse {
return AddCronResponse.fromPartial(base ?? {});
return AddCronResponse.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<AddCronResponse>, I>>(_: I): AddCronResponse {
const message = createBaseAddCronResponse();
return message;
@@ -321,55 +336,53 @@ function createBaseDeleteCronRequest(): DeleteCronRequest {
return { ids: [] };
}
export const DeleteCronRequest = {
encode(message: DeleteCronRequest, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const DeleteCronRequest: MessageFns<DeleteCronRequest> = {
encode(message: DeleteCronRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
for (const v of message.ids) {
writer.uint32(10).string(v!);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): DeleteCronRequest {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): DeleteCronRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseDeleteCronRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.ids.push(reader.string());
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): DeleteCronRequest {
return { ids: Array.isArray(object?.ids) ? object.ids.map((e: any) => String(e)) : [] };
return { ids: globalThis.Array.isArray(object?.ids) ? object.ids.map((e: any) => globalThis.String(e)) : [] };
},
toJSON(message: DeleteCronRequest): unknown {
const obj: any = {};
if (message.ids) {
obj.ids = message.ids.map((e) => e);
} else {
obj.ids = [];
if (message.ids?.length) {
obj.ids = message.ids;
}
return obj;
},
create<I extends Exact<DeepPartial<DeleteCronRequest>, I>>(base?: I): DeleteCronRequest {
return DeleteCronRequest.fromPartial(base ?? {});
return DeleteCronRequest.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<DeleteCronRequest>, I>>(object: I): DeleteCronRequest {
const message = createBaseDeleteCronRequest();
message.ids = object.ids?.map((e) => e) || [];
@@ -381,13 +394,13 @@ function createBaseDeleteCronResponse(): DeleteCronResponse {
return {};
}
export const DeleteCronResponse = {
encode(_: DeleteCronResponse, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const DeleteCronResponse: MessageFns<DeleteCronResponse> = {
encode(_: DeleteCronResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): DeleteCronResponse {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): DeleteCronResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseDeleteCronResponse();
while (reader.pos < end) {
@@ -397,7 +410,7 @@ export const DeleteCronResponse = {
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -412,9 +425,8 @@ export const DeleteCronResponse = {
},
create<I extends Exact<DeepPartial<DeleteCronResponse>, I>>(base?: I): DeleteCronResponse {
return DeleteCronResponse.fromPartial(base ?? {});
return DeleteCronResponse.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<DeleteCronResponse>, I>>(_: I): DeleteCronResponse {
const message = createBaseDeleteCronResponse();
return message;
@@ -484,12 +496,14 @@ export interface CronClient extends Client {
export const CronClient = makeGenericClientConstructor(CronService, "com.ql.cron.Cron") as unknown as {
new (address: string, credentials: ChannelCredentials, options?: Partial<ClientOptions>): CronClient;
service: typeof CronService;
serviceName: string;
};
type Builtin = Date | Function | Uint8Array | string | number | boolean | undefined;
export type DeepPartial<T> = T extends Builtin ? T
: T extends Array<infer U> ? Array<DeepPartial<U>> : T extends ReadonlyArray<infer U> ? ReadonlyArray<DeepPartial<U>>
: T extends globalThis.Array<infer U> ? globalThis.Array<DeepPartial<U>>
: T extends ReadonlyArray<infer U> ? ReadonlyArray<DeepPartial<U>>
: T extends {} ? { [K in keyof T]?: DeepPartial<T[K]> }
: Partial<T>;
@@ -500,3 +514,12 @@ export type Exact<P, I extends P> = P extends Builtin ? P
function isSet(value: any): boolean {
return value !== null && value !== undefined;
}
export interface MessageFns<T> {
encode(message: T, writer?: BinaryWriter): BinaryWriter;
decode(input: BinaryReader | Uint8Array, length?: number): T;
fromJSON(object: any): T;
toJSON(message: T): unknown;
create<I extends Exact<DeepPartial<T>, I>>(base?: I): T;
fromPartial<I extends Exact<DeepPartial<T>, I>>(object: I): T;
}
+50 -29
View File
@@ -1,19 +1,25 @@
// Code generated by protoc-gen-ts_proto. DO NOT EDIT.
// versions:
// protoc-gen-ts_proto v2.6.1
// protoc v3.21.12
// source: back/protos/health.proto
/* eslint-disable */
import { BinaryReader, BinaryWriter } from "@bufbuild/protobuf/wire";
import {
CallOptions,
type CallOptions,
ChannelCredentials,
Client,
ClientOptions,
ClientReadableStream,
ClientUnaryCall,
handleServerStreamingCall,
handleUnaryCall,
type ClientOptions,
type ClientReadableStream,
type ClientUnaryCall,
type handleServerStreamingCall,
type handleUnaryCall,
makeGenericClientConstructor,
Metadata,
ServiceError,
UntypedServiceImplementation,
type ServiceError,
type UntypedServiceImplementation,
} from "@grpc/grpc-js";
import _m0 from "protobufjs/minimal";
export const protobufPackage = "com.ql.health";
@@ -74,51 +80,53 @@ function createBaseHealthCheckRequest(): HealthCheckRequest {
return { service: "" };
}
export const HealthCheckRequest = {
encode(message: HealthCheckRequest, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const HealthCheckRequest: MessageFns<HealthCheckRequest> = {
encode(message: HealthCheckRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.service !== "") {
writer.uint32(10).string(message.service);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): HealthCheckRequest {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): HealthCheckRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseHealthCheckRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.service = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): HealthCheckRequest {
return { service: isSet(object.service) ? String(object.service) : "" };
return { service: isSet(object.service) ? globalThis.String(object.service) : "" };
},
toJSON(message: HealthCheckRequest): unknown {
const obj: any = {};
message.service !== undefined && (obj.service = message.service);
if (message.service !== "") {
obj.service = message.service;
}
return obj;
},
create<I extends Exact<DeepPartial<HealthCheckRequest>, I>>(base?: I): HealthCheckRequest {
return HealthCheckRequest.fromPartial(base ?? {});
return HealthCheckRequest.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<HealthCheckRequest>, I>>(object: I): HealthCheckRequest {
const message = createBaseHealthCheckRequest();
message.service = object.service ?? "";
@@ -130,33 +138,34 @@ function createBaseHealthCheckResponse(): HealthCheckResponse {
return { status: 0 };
}
export const HealthCheckResponse = {
encode(message: HealthCheckResponse, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const HealthCheckResponse: MessageFns<HealthCheckResponse> = {
encode(message: HealthCheckResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.status !== 0) {
writer.uint32(8).int32(message.status);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): HealthCheckResponse {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): HealthCheckResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseHealthCheckResponse();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 8) {
break;
}
message.status = reader.int32() as any;
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -167,14 +176,15 @@ export const HealthCheckResponse = {
toJSON(message: HealthCheckResponse): unknown {
const obj: any = {};
message.status !== undefined && (obj.status = healthCheckResponse_ServingStatusToJSON(message.status));
if (message.status !== 0) {
obj.status = healthCheckResponse_ServingStatusToJSON(message.status);
}
return obj;
},
create<I extends Exact<DeepPartial<HealthCheckResponse>, I>>(base?: I): HealthCheckResponse {
return HealthCheckResponse.fromPartial(base ?? {});
return HealthCheckResponse.fromPartial(base ?? ({} as any));
},
fromPartial<I extends Exact<DeepPartial<HealthCheckResponse>, I>>(object: I): HealthCheckResponse {
const message = createBaseHealthCheckResponse();
message.status = object.status ?? 0;
@@ -236,12 +246,14 @@ export interface HealthClient extends Client {
export const HealthClient = makeGenericClientConstructor(HealthService, "com.ql.health.Health") as unknown as {
new (address: string, credentials: ChannelCredentials, options?: Partial<ClientOptions>): HealthClient;
service: typeof HealthService;
serviceName: string;
};
type Builtin = Date | Function | Uint8Array | string | number | boolean | undefined;
export type DeepPartial<T> = T extends Builtin ? T
: T extends Array<infer U> ? Array<DeepPartial<U>> : T extends ReadonlyArray<infer U> ? ReadonlyArray<DeepPartial<U>>
: T extends globalThis.Array<infer U> ? globalThis.Array<DeepPartial<U>>
: T extends ReadonlyArray<infer U> ? ReadonlyArray<DeepPartial<U>>
: T extends {} ? { [K in keyof T]?: DeepPartial<T[K]> }
: Partial<T>;
@@ -252,3 +264,12 @@ export type Exact<P, I extends P> = P extends Builtin ? P
function isSet(value: any): boolean {
return value !== null && value !== undefined;
}
export interface MessageFns<T> {
encode(message: T, writer?: BinaryWriter): BinaryWriter;
decode(input: BinaryReader | Uint8Array, length?: number): T;
fromJSON(object: any): T;
toJSON(message: T): unknown;
create<I extends Exact<DeepPartial<T>, I>>(base?: I): T;
fromPartial<I extends Exact<DeepPartial<T>, I>>(object: I): T;
}
-36
View File
@@ -1,36 +0,0 @@
import express from 'express';
import Logger from './loaders/logger';
import config from './config';
import { HealthClient } from './protos/health';
import { credentials } from '@grpc/grpc-js';
const app = express();
const client = new HealthClient(
`0.0.0.0:${config.cronPort}`,
credentials.createInsecure(),
{ 'grpc.enable_http_proxy': 0 },
);
app.get('/api/health', (req, res) => {
client.check({ service: 'cron' }, (err, response) => {
if (err) {
return res.status(200).send({ code: 500, error: err });
}
return res.status(200).send({ code: 200, data: response });
});
});
app
.listen(config.publicPort, async () => {
await require('./loaders/sentry').default({ expressApp: app });
await require('./loaders/db').default();
Logger.debug(`✌️ 公共服务启动成功!`);
console.debug(`✌️ 公共服务启动成功!`);
process.send?.('ready');
})
.on('error', (err) => {
Logger.error(err);
console.error(err);
process.exit(1);
});
+14 -14
View File
@@ -10,43 +10,43 @@ const addCron = (
callback: sendUnaryData<AddCronResponse>,
) => {
for (const item of call.request.crons) {
const { id, schedule, command, extraSchedules, name } = item;
const { id, schedule, command, extra_schedules, name } = item;
if (scheduleStacks.has(id)) {
scheduleStacks.get(id)?.forEach((x) => x.cancel());
}
Logger.info(
'[schedule][创建定时任务], 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
'[schedule][创建定时任务] 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
id,
name,
schedule,
command,
);
if (extraSchedules?.length) {
extraSchedules.forEach(x => {
if (extra_schedules?.length) {
extra_schedules.forEach((x) => {
Logger.info(
'[schedule][创建定时任务], 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
'[schedule][创建定时任务] 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
id,
name,
x.schedule,
command,
);
})
});
}
scheduleStacks.set(id, [
nodeSchedule.scheduleJob(id, schedule, async () => {
Logger.info(`[schedule][准备运行任务] 命令: ${command}`);
runCron(command, { name, schedule, extraSchedules });
runCron(command, item);
}),
...(extraSchedules?.length
? extraSchedules.map((x) =>
nodeSchedule.scheduleJob(id, x.schedule, async () => {
Logger.info(`[schedule][准备运行任务] 命令: ${command}`);
runCron(command, { name, schedule, extraSchedules });
}),
)
...(extra_schedules?.length
? extra_schedules.map((x) =>
nodeSchedule.scheduleJob(id, x.schedule, async () => {
Logger.info(`[schedule][准备运行任务] 命令: ${command}`);
runCron(command, item);
}),
)
: []),
]);
}
+444
View File
@@ -0,0 +1,444 @@
import 'reflect-metadata';
import { Container } from 'typedi';
import EnvService from '../services/env';
import { sendUnaryData, ServerUnaryCall } from '@grpc/grpc-js';
import {
CreateEnvRequest,
CronItem,
DeleteEnvsRequest,
DisableEnvsRequest,
EnableEnvsRequest,
EnvItem,
EnvResponse,
EnvsResponse,
GetEnvByIdRequest,
GetEnvsRequest,
MoveEnvRequest,
Response,
SystemNotifyRequest,
UpdateEnvNamesRequest,
UpdateEnvRequest,
} from '../protos/api';
import LoggerInstance from '../loaders/logger';
import pick from 'lodash/pick';
import SystemService from '../services/system';
import CronService from '../services/cron';
import {
CronDetailRequest,
CronDetailResponse,
CreateCronRequest,
UpdateCronRequest,
DeleteCronsRequest,
CronResponse,
GetCronsRequest,
CronsResponse,
GetCronByIdRequest,
EnableCronsRequest,
DisableCronsRequest,
RunCronsRequest,
} from '../protos/api';
import { NotificationInfo } from '../data/notify';
Container.set('logger', LoggerInstance);
export const getEnvs = async (
call: ServerUnaryCall<GetEnvsRequest, EnvsResponse>,
callback: sendUnaryData<EnvsResponse>,
) => {
try {
const envService = Container.get(EnvService);
const data = await envService.envs(call.request.searchValue);
callback(null, {
code: 200,
data: data.map((x) => ({ ...x, remarks: x.remarks || '' })),
});
} catch (e: any) {
callback(null, {
code: 500,
data: [],
message: e.message,
});
}
};
export const createEnv = async (
call: ServerUnaryCall<CreateEnvRequest, EnvsResponse>,
callback: sendUnaryData<EnvsResponse>,
) => {
try {
const envService = Container.get(EnvService);
const data = await envService.create(call.request.envs);
callback(null, { code: 200, data });
} catch (e: any) {
callback(e);
}
};
export const updateEnv = async (
call: ServerUnaryCall<UpdateEnvRequest, EnvResponse>,
callback: sendUnaryData<EnvResponse>,
) => {
try {
if (!call.request.env?.id) {
return callback(null, {
code: 400,
data: undefined,
message: 'id parameter is required',
});
}
const envService = Container.get(EnvService);
const data = await envService.update(
pick(call.request.env, ['id', 'name', 'value', 'remarks']) as EnvItem,
);
callback(null, { code: 200, data });
} catch (e: any) {
callback(e);
}
};
export const deleteEnvs = async (
call: ServerUnaryCall<DeleteEnvsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.remove(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
export const moveEnv = async (
call: ServerUnaryCall<MoveEnvRequest, EnvResponse>,
callback: sendUnaryData<EnvResponse>,
) => {
try {
if (!call.request.id) {
return callback(null, {
code: 400,
data: undefined,
message: 'id parameter is required',
});
}
const envService = Container.get(EnvService);
const data = await envService.move(call.request.id, {
fromIndex: call.request.fromIndex,
toIndex: call.request.toIndex,
});
callback(null, { code: 200, data });
} catch (e: any) {
callback(e);
}
};
export const disableEnvs = async (
call: ServerUnaryCall<DisableEnvsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.disabled(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
export const enableEnvs = async (
call: ServerUnaryCall<EnableEnvsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.enabled(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
export const updateEnvNames = async (
call: ServerUnaryCall<UpdateEnvNamesRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.updateNames({
ids: call.request.ids,
name: call.request.name,
});
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
export const getEnvById = async (
call: ServerUnaryCall<GetEnvByIdRequest, EnvResponse>,
callback: sendUnaryData<EnvResponse>,
) => {
try {
if (!call.request.id) {
return callback(null, {
code: 400,
data: undefined,
message: 'id parameter is required',
});
}
const envService = Container.get(EnvService);
const data = await envService.getDb({ id: call.request.id });
callback(null, {
code: 200,
data: { ...data, remarks: data.remarks || '' },
});
} catch (e: any) {
callback(e);
}
};
export const systemNotify = async (
call: ServerUnaryCall<SystemNotifyRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
const systemService = Container.get(SystemService);
const data = await systemService.notify({
title: call.request.title,
content: call.request.content,
notificationInfo: call.request.notificationInfo as unknown as NotificationInfo,
});
callback(null, data);
} catch (e: any) {
callback(e);
}
};
const normalizeCronData = (data: CronItem | null): CronItem | undefined => {
if (!data) return undefined;
return {
...data,
sub_id: data.sub_id ?? undefined,
extra_schedules: data.extra_schedules ?? undefined,
pid: data.pid ?? undefined,
task_before: data.task_before ?? undefined,
task_after: data.task_after ?? undefined,
};
};
export const getCronDetail = async (
call: ServerUnaryCall<CronDetailRequest, CronDetailResponse>,
callback: sendUnaryData<CronDetailResponse>,
) => {
try {
if (!call.request.log_path) {
return callback(null, {
code: 400,
data: undefined,
message: 'log_path is required',
});
}
const cronService = Container.get(CronService);
const data = (await cronService.find({
log_path: call.request.log_path,
})) as CronItem;
callback(null, { code: 200, data: normalizeCronData(data) });
} catch (e: any) {
callback(e);
}
};
export const createCron = async (
call: ServerUnaryCall<CreateCronRequest, CronResponse>,
callback: sendUnaryData<CronResponse>,
) => {
try {
const cronService = Container.get(CronService);
const data = (await cronService.create(call.request)) as CronItem;
callback(null, { code: 200, data: normalizeCronData(data) });
} catch (e: any) {
callback(e);
}
};
export const updateCron = async (
call: ServerUnaryCall<UpdateCronRequest, CronResponse>,
callback: sendUnaryData<CronResponse>,
) => {
try {
const cronService = Container.get(CronService);
const { id, ...fields } = call.request;
const updateRequest = {
id,
...Object.entries(fields).reduce((acc: any, [key, value]) => {
if (value !== undefined) {
acc[key] = value;
}
return acc;
}, {}),
} as UpdateCronRequest;
const data = (await cronService.update(updateRequest)) as CronItem;
callback(null, { code: 200, data: normalizeCronData(data) });
} catch (e: any) {
callback(e);
}
};
export const deleteCrons = async (
call: ServerUnaryCall<DeleteCronsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
const cronService = Container.get(CronService);
await cronService.remove(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
export const getCrons = async (
call: ServerUnaryCall<GetCronsRequest, CronsResponse>,
callback: sendUnaryData<CronsResponse>,
) => {
try {
const cronService = Container.get(CronService);
const result = await cronService.crontabs({
searchValue: call.request.searchValue || '',
page: '0',
size: '0',
sorter: '',
filters: '',
queryString: '',
});
const data = result.data.map((x) => normalizeCronData(x as CronItem));
callback(null, {
code: 200,
data: data.filter((x): x is CronItem => x !== undefined),
});
} catch (e: any) {
callback(null, {
code: 500,
data: [],
message: e.message,
});
}
};
export const getCronById = async (
call: ServerUnaryCall<GetCronByIdRequest, CronResponse>,
callback: sendUnaryData<CronResponse>,
) => {
try {
if (!call.request.id) {
return callback(null, {
code: 400,
data: undefined,
message: 'id parameter is required',
});
}
const cronService = Container.get(CronService);
const data = (await cronService.getDb({ id: call.request.id })) as CronItem;
callback(null, { code: 200, data: normalizeCronData(data) });
} catch (e: any) {
callback(null, {
code: 404,
data: undefined,
message: e.message,
});
}
};
export const enableCrons = async (
call: ServerUnaryCall<EnableCronsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const cronService = Container.get(CronService);
await cronService.enabled(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
export const disableCrons = async (
call: ServerUnaryCall<DisableCronsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const cronService = Container.get(CronService);
await cronService.disabled(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
export const runCrons = async (
call: ServerUnaryCall<RunCronsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const cronService = Container.get(CronService);
await cronService.run(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
+1 -1
View File
@@ -10,7 +10,7 @@ import config from '../config';
class Client {
private client = new CronClient(
`0.0.0.0:${config.cronPort}`,
`0.0.0.0:${config.grpcPort}`,
credentials.createInsecure(),
{ 'grpc.enable_http_proxy': 0 },
);
+1 -1
View File
@@ -10,7 +10,7 @@ const delCron = (
for (const id of call.request.ids) {
if (scheduleStacks.has(id)) {
Logger.info(
'[schedule][取消定时任务], 任务ID: %s',
'[schedule][取消定时任务] 任务ID: %s',
id,
);
scheduleStacks.get(id)?.forEach(x => x.cancel());
+3 -6
View File
@@ -17,14 +17,11 @@ const check = async (
return callback(null, { status: 1 });
}
const panelErrLog = await promiseExec(
`tail -n 300 ~/.pm2/logs/panel-error.log`,
);
const scheduleErrLog = await promiseExec(
`tail -n 300 ~/.pm2/logs/schedule-error.log`,
const qinglongErrLog = await promiseExec(
`tail -n 300 ~/.pm2/logs/qinglong-error.log`,
);
return callback(
new Error(`${scheduleErrLog || ''}\n${panelErrLog || ''}\n${res}`.trim()),
new Error(`${qinglongErrLog || ''}\n${res}`.trim()),
);
default:
-25
View File
@@ -1,25 +0,0 @@
import { Server, ServerCredentials } from '@grpc/grpc-js';
import { CronService } from '../protos/cron';
import { addCron } from './addCron';
import { delCron } from './delCron';
import { HealthService } from '../protos/health';
import { check } from './health';
import config from '../config';
import Logger from '../loaders/logger';
const server = new Server({ 'grpc.enable_http_proxy': 0 });
server.addService(HealthService, { check });
server.addService(CronService, { addCron, delCron });
server.bindAsync(
`0.0.0.0:${config.cronPort}`,
ServerCredentials.createInsecure(),
(err, port) => {
if (err) {
throw err;
}
server.start();
Logger.debug(`✌️ 定时服务启动成功!`);
console.debug(`✌️ 定时服务启动成功!`);
process.send?.('ready');
},
);
+17 -5
View File
@@ -3,7 +3,7 @@ import path, { join } from 'path';
import config from '../config';
import { getFileContentByName } from '../config/util';
import { Response } from 'express';
import got from 'got';
import { request } from 'undici';
@Service()
export default class ConfigService {
@@ -11,14 +11,26 @@ export default class ConfigService {
public async getFile(filePath: string, res: Response) {
let content = '';
if (config.blackFileList.includes(filePath) || !filePath) {
res.send({ code: 403, message: '文件无法访问' });
const avaliablePath = [config.rootPath, config.configPath].map((x) =>
path.resolve(x, filePath),
);
if (
config.blackFileList.includes(filePath) ||
avaliablePath.every(
(x) =>
!x.startsWith(config.scriptPath) && !x.startsWith(config.configPath),
) ||
!filePath
) {
return res.send({ code: 403, message: '文件无法访问' });
}
if (filePath.startsWith('sample/')) {
const res = await got.get(
const res = await request(
`https://gitlab.com/whyour/qinglong/-/raw/master/${filePath}`,
);
content = res.body;
content = await res.body.text();
} else if (filePath.startsWith('data/scripts/')) {
content = await getFileContentByName(join(config.rootPath, filePath));
} else {
+228 -72
View File
@@ -4,15 +4,17 @@ import config from '../config';
import { Crontab, CrontabModel, CrontabStatus } from '../data/cron';
import { exec, execSync } from 'child_process';
import fs from 'fs/promises';
import cron_parser from 'cron-parser';
import CronExpressionParser from 'cron-parser';
import {
getFileContentByName,
fileExist,
killTask,
killAllTasks,
getUniqPath,
safeJSONParse,
isDemoEnv,
} from '../config/util';
import { Op, where, col as colFn, FindOptions, fn } from 'sequelize';
import { Op, where, col as colFn, FindOptions, fn, Order } from 'sequelize';
import path from 'path';
import { TASK_PREFIX, QL_PREFIX } from '../config/const';
import cronClient from '../schedule/client';
@@ -21,10 +23,13 @@ import { spawn } from 'cross-spawn';
import dayjs from 'dayjs';
import pickBy from 'lodash/pickBy';
import omit from 'lodash/omit';
import { writeFileWithLock } from '../shared/utils';
import { ScheduleType } from '../interface/schedule';
import { logStreamManager } from '../shared/logStreamManager';
@Service()
export default class CronService {
constructor(@Inject('logger') private logger: winston.Logger) {}
constructor(@Inject('logger') private logger: winston.Logger) { }
private isNodeCron(cron: Crontab) {
const { schedule, extra_schedules } = cron;
@@ -34,22 +39,60 @@ export default class CronService {
return false;
}
private isOnceSchedule(schedule?: string) {
return schedule?.startsWith(ScheduleType.ONCE);
}
private isBootSchedule(schedule?: string) {
return schedule?.startsWith(ScheduleType.BOOT);
}
private isSpecialSchedule(schedule?: string) {
return this.isOnceSchedule(schedule) || this.isBootSchedule(schedule);
}
private async getLogName(cron: Crontab) {
const { log_name, command, id } = cron;
if (log_name === '/dev/null') {
return log_name;
}
let uniqPath = await getUniqPath(command, `${id}`);
if (log_name) {
const normalizedLogName = log_name.startsWith('/')
? log_name
: path.join(config.logPath, log_name);
if (normalizedLogName.startsWith(config.logPath)) {
uniqPath = log_name;
}
}
const logDirPath = path.resolve(config.logPath, `${uniqPath}`);
await fs.mkdir(logDirPath, { recursive: true });
return uniqPath;
}
public async create(payload: Crontab): Promise<Crontab> {
const tab = new Crontab(payload);
tab.saved = false;
tab.log_name = await this.getLogName(tab);
const doc = await this.insert(tab);
if (this.isNodeCron(doc)) {
if (isDemoEnv()) {
return doc;
}
if (this.isNodeCron(doc) && !this.isSpecialSchedule(doc.schedule)) {
await cronClient.addCron([
{
name: doc.name || '',
id: String(doc.id),
schedule: doc.schedule!,
command: this.makeCommand(doc),
extraSchedules: doc.extra_schedules || [],
extra_schedules: doc.extra_schedules || [],
},
]);
}
await this.set_crontab();
await this.setCrontab();
return doc;
}
@@ -57,29 +100,34 @@ export default class CronService {
return await CrontabModel.create(payload, { returning: true });
}
public async update(payload: Crontab): Promise<Crontab> {
public async update(payload: Partial<Crontab>): Promise<Crontab> {
const doc = await this.getDb({ id: payload.id });
const tab = new Crontab({ ...doc, ...payload });
tab.saved = false;
tab.log_name = await this.getLogName(tab);
const newDoc = await this.updateDb(tab);
if (doc.isDisabled === 1) {
if (doc.isDisabled === 1 || isDemoEnv()) {
return newDoc;
}
if (this.isNodeCron(doc)) {
await cronClient.delCron([String(doc.id)]);
}
if (this.isNodeCron(newDoc)) {
if (this.isNodeCron(newDoc) && !this.isSpecialSchedule(newDoc.schedule)) {
await cronClient.addCron([
{
name: doc.name || '',
id: String(newDoc.id),
schedule: newDoc.schedule!,
command: this.makeCommand(newDoc),
extraSchedules: newDoc.extra_schedules || [],
extra_schedules: newDoc.extra_schedules || [],
},
]);
}
await this.set_crontab();
await this.setCrontab();
return newDoc;
}
@@ -114,7 +162,13 @@ export default class CronService {
}
for (const id of ids) {
const cron = await this.getDb({ id });
let cron;
try {
cron = await this.getDb({ id });
} catch (err) { }
if (!cron) {
continue;
}
if (status === CrontabStatus.idle && log_path !== cron.log_path) {
options = omit(options, ['status', 'log_path', 'pid']);
}
@@ -128,7 +182,7 @@ export default class CronService {
public async remove(ids: number[]) {
await CrontabModel.destroy({ where: { id: ids } });
await cronClient.delCron(ids.map(String));
await this.set_crontab();
await this.setCrontab();
}
public async pin(ids: number[]) {
@@ -172,8 +226,8 @@ export default class CronService {
for (const col of viewQuery.filters) {
const { property, value, operation } = col;
let q: any = {};
let operate2 = null;
let operate = null;
let operate2: any = null;
let operate: any = null;
switch (operation) {
case 'Reg':
operate = Op.like;
@@ -184,21 +238,38 @@ export default class CronService {
operate2 = Op.and;
break;
case 'In':
q[Op.or] = [
{
[property]: Array.isArray(value) ? value : [value],
},
property === 'status' && value.includes(2)
? { isDisabled: 1 }
: {},
];
if (
property === 'status' &&
!value.includes(CrontabStatus.disabled)
) {
q[Op.and] = [
{ [property]: Array.isArray(value) ? value : [value] },
{ isDisabled: 0 },
];
} else {
q[Op.or] = [
{
[property]: Array.isArray(value) ? value : [value],
},
property === 'status' && value.includes(CrontabStatus.disabled)
? { isDisabled: 1 }
: {},
];
}
break;
case 'Nin':
q[Op.and] = [
{
[property]: {
[Op.notIn]: Array.isArray(value) ? value : [value],
},
[Op.or]: [
{
[property]: {
[Op.notIn]: Array.isArray(value) ? value : [value],
},
},
{
[property]: { [Op.is]: null },
},
],
},
property === 'status' && value.includes(2)
? { isDisabled: { [Op.ne]: 1 } }
@@ -313,10 +384,10 @@ export default class CronService {
log_path,
}: {
log_path: string;
}): Promise<Crontab | null> {
}): Promise<Crontab | undefined> {
try {
const result = await CrontabModel.findOne({ where: { log_path } });
return result;
return result?.get({ plain: true });
} catch (error) {
throw error;
}
@@ -356,9 +427,9 @@ export default class CronService {
order.unshift([field, type]);
}
}
let condition: any = {
let condition: FindOptions<Crontab> = {
where: query,
order: order,
order: order as Order,
};
if (page && size) {
condition.offset = (page - 1) * size;
@@ -367,7 +438,7 @@ export default class CronService {
try {
const result = await CrontabModel.findAll(condition);
const count = await CrontabModel.count({ where: query });
return { data: result, total: count };
return { data: result.map((x) => x.get({ plain: true })), total: count };
} catch (error) {
throw error;
}
@@ -375,7 +446,10 @@ export default class CronService {
public async getDb(query: FindOptions<Crontab>['where']): Promise<Crontab> {
const doc: any = await CrontabModel.findOne({ where: { ...query } });
return doc && (doc.get({ plain: true }) as Crontab);
if (!doc) {
throw new Error(`Cron ${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
public async run(ids: number[]) {
@@ -391,12 +465,20 @@ export default class CronService {
public async stop(ids: number[]) {
const docs = await CrontabModel.findAll({ where: { id: ids } });
for (const doc of docs) {
if (doc.pid) {
try {
// Kill all running instances of this task
try {
if (doc.pid) {
await killTask(doc.pid);
} catch (error) {
this.logger.error(error);
}
const command = doc.command.replace(/\s+/g, ' ').trim();
await killAllTasks(command);
this.logger.info(
`[panel][停止所有运行中的任务实例] 任务ID: ${doc.id}, 命令: ${command}`,
);
} catch (error) {
this.logger.error(
`[panel][停止任务失败] 任务ID: ${doc.id}, 错误: ${error}`,
);
}
}
@@ -407,14 +489,14 @@ export default class CronService {
}
private async runSingle(cronId: number): Promise<number | void> {
return taskLimit.runWithCronLimit(() => {
return taskLimit.manualRunWithCronLimit(() => {
return new Promise(async (resolve: any) => {
const cron = await this.getDb({ id: cronId });
const params = {
name: cron.name,
command: cron.command,
schedule: cron.schedule,
extraSchedules: cron.extra_schedules,
extra_schedules: cron.extra_schedules,
};
if (cron.status !== CrontabStatus.queued) {
resolve(params);
@@ -422,21 +504,25 @@ export default class CronService {
}
this.logger.info(
`[panel][开始执行任务] 参数 ${JSON.stringify(params)}`,
`[panel][开始执行任务] 参数: ${JSON.stringify(params)}`,
);
let { id, command, log_path } = cron;
const uniqPath = await getUniqPath(command, `${id}`);
let { id, command, log_name } = cron;
const uniqPath =
log_name === '/dev/null' || !log_name
? await getUniqPath(command, `${id}`)
: log_name;
const logTime = dayjs().format('YYYY-MM-DD-HH-mm-ss-SSS');
const logDirPath = path.resolve(config.logPath, `${uniqPath}`);
if (log_path?.split('/')?.every((x) => x !== uniqPath)) {
await fs.mkdir(logDirPath, { recursive: true });
}
await fs.mkdir(logDirPath, { recursive: true });
const logPath = `${uniqPath}/${logTime}.log`;
const absolutePath = path.resolve(config.logPath, `${logPath}`);
const cp = spawn(
`real_log_path=${logPath} no_delay=true ${this.makeCommand(cron)}`,
`real_log_path=${logPath} no_delay=true ${this.makeCommand(
cron,
true,
)}`,
{ shell: '/bin/bash' },
);
@@ -444,14 +530,34 @@ export default class CronService {
{ status: CrontabStatus.running, pid: cp.pid, log_path: logPath },
{ where: { id } },
);
cp.stdout.on('data', async (data) => {
await logStreamManager.write(absolutePath, data.toString());
});
cp.stderr.on('data', async (data) => {
await fs.appendFile(`${absolutePath}`, `${data.toString()}`);
this.logger.info(
'[panel][执行任务失败] 命令: %s, 错误信息: %j',
command,
data.toString(),
);
await logStreamManager.write(absolutePath, data.toString());
});
cp.on('error', async (err) => {
await fs.appendFile(`${absolutePath}`, `${JSON.stringify(err)}`);
this.logger.error(
'[panel][创建任务失败] 命令: %s, 错误信息: %j',
command,
err,
);
await logStreamManager.write(absolutePath, JSON.stringify(err));
});
cp.on('exit', async (code) => {
this.logger.info(
'[panel][执行任务结束] 参数: %s, 退出码: %j',
JSON.stringify(params),
code,
);
// Close the stream after task completion
await logStreamManager.closeStream(absolutePath);
await CrontabModel.update(
{ status: CrontabStatus.idle, pid: undefined },
{ where: { id } },
@@ -465,23 +571,27 @@ export default class CronService {
public async disabled(ids: number[]) {
await CrontabModel.update({ isDisabled: 1 }, { where: { id: ids } });
await cronClient.delCron(ids.map(String));
await this.set_crontab();
await this.setCrontab();
}
public async enabled(ids: number[]) {
await CrontabModel.update({ isDisabled: 0 }, { where: { id: ids } });
const docs = await CrontabModel.findAll({ where: { id: ids } });
const sixCron = docs
.filter((x) => this.isNodeCron(x))
.filter((x) => this.isNodeCron(x) && !this.isSpecialSchedule(x.schedule))
.map((doc) => ({
name: doc.name || '',
id: String(doc.id),
schedule: doc.schedule!,
command: this.makeCommand(doc),
extraSchedules: doc.extra_schedules || [],
extra_schedules: doc.extra_schedules || [],
}));
if (isDemoEnv()) {
return;
}
await cronClient.addCron(sixCron);
await this.set_crontab();
await this.setCrontab();
}
public async log(id: number) {
@@ -489,13 +599,18 @@ export default class CronService {
if (!doc) {
return '';
}
if (doc.log_name === '/dev/null') {
return '日志设置为忽略';
}
const absolutePath = path.resolve(config.logPath, `${doc.log_path}`);
const logFileExist = doc.log_path && (await fileExist(absolutePath));
if (logFileExist) {
return await getFileContentByName(`${absolutePath}`);
} else {
return '任务未运行';
return typeof doc.status === 'number' &&
[CrontabStatus.queued, CrontabStatus.running].includes(doc.status)
? '运行中...'
: '日志不存在...';
}
}
@@ -515,7 +630,7 @@ export default class CronService {
files.map(async (x) => ({
filename: x,
directory: relativeDir.replace(config.logPath, ''),
time: (await fs.stat(`${dir}/${x}`)).mtime.getTime(),
time: (await fs.lstat(`${dir}/${x}`)).birthtimeMs,
})),
)
).sort((a, b) => b.time - a.time);
@@ -524,20 +639,26 @@ export default class CronService {
}
}
private makeCommand(tab: Crontab) {
private makeCommand(tab: Crontab, realTime?: boolean) {
let command = tab.command.trim();
if (!command.startsWith(TASK_PREFIX) && !command.startsWith(QL_PREFIX)) {
command = `${TASK_PREFIX}${tab.command}`;
}
let commandVariable = `no_tee=true ID=${tab.id} `;
let commandVariable = `real_time=${Boolean(realTime)} no_tee=true ID=${tab.id} `;
// Only include log_name if it has a truthy value to avoid passing null/undefined to shell
if (tab.log_name) {
commandVariable += `log_name=${tab.log_name} `;
}
if (tab.task_before) {
commandVariable += `task_before='${tab.task_before
.replace(/'/g, "'\\''")
.replace(/;? *\n/g, ';')
.trim()}' `;
}
if (tab.task_after) {
commandVariable += `task_after='${tab.task_after
.replace(/'/g, "'\\''")
.replace(/;? *\n/g, ';')
.trim()}' `;
}
@@ -545,15 +666,14 @@ export default class CronService {
return crontab_job_string;
}
private async set_crontab(data?: { data: Crontab[]; total: number }) {
private async setCrontab(data?: { data: Crontab[]; total: number }) {
const tabs = data ?? (await this.crontabs());
var crontab_string = '';
tabs.data.forEach((tab) => {
const _schedule = tab.schedule && tab.schedule.split(/ +/);
if (
tab.isDisabled === 1 ||
_schedule!.length !== 5 ||
tab.extra_schedules?.length
this.isNodeCron(tab) ||
this.isSpecialSchedule(tab.schedule)
) {
crontab_string += '# ';
crontab_string += tab.schedule;
@@ -568,14 +688,25 @@ export default class CronService {
}
});
await fs.writeFile(config.crontabFile, crontab_string);
await writeFileWithLock(config.crontabFile, crontab_string);
try {
execSync(`crontab ${config.crontabFile}`);
} catch (error: any) {
const errorMsg = error.message || String(error);
this.logger.error('[crontab] Failed to update system crontab:', errorMsg);
}
execSync(`crontab ${config.crontabFile}`);
await CrontabModel.update({ saved: true }, { where: {} });
}
public import_crontab() {
exec('crontab -l', (error, stdout, stderr) => {
public importCrontab() {
exec('crontab -l', (error, stdout) => {
if (error) {
const errorMsg = error.message || String(error);
this.logger.error('[crontab] Failed to read system crontab:', errorMsg);
}
const lines = stdout.split('\n');
const namePrefix = new Date().getTime();
@@ -589,7 +720,7 @@ export default class CronService {
if (
command &&
schedule &&
cron_parser.parseExpression(schedule).hasNext()
CronExpressionParser.parse(schedule).hasNext()
) {
const name = namePrefix + '_' + index;
@@ -610,17 +741,42 @@ export default class CronService {
public async autosave_crontab() {
const tabs = await this.crontabs();
this.set_crontab(tabs);
const sixCron = tabs.data
.filter((x) => this.isNodeCron(x) && x.isDisabled !== 1)
const regularCrons = tabs.data
.filter(
(x) =>
x.isDisabled !== 1 &&
this.isNodeCron(x) &&
!this.isSpecialSchedule(x.schedule),
)
.map((doc) => ({
name: doc.name || '',
id: String(doc.id),
schedule: doc.schedule!,
command: this.makeCommand(doc),
extraSchedules: doc.extra_schedules || [],
extra_schedules: doc.extra_schedules || [],
}));
await cronClient.addCron(sixCron);
if (isDemoEnv()) {
await writeFileWithLock(config.crontabFile, '');
return;
}
await cronClient.addCron(regularCrons);
this.setCrontab(tabs);
}
public async bootTask() {
const tabs = await this.crontabs();
const bootTasks = tabs.data.filter(
(x) => !x.isDisabled && this.isBootSchedule(x.schedule),
);
if (bootTasks.length > 0) {
await CrontabModel.update(
{ status: CrontabStatus.queued },
{ where: { id: bootTasks.map((t) => t.id!) } },
);
for (const task of bootTasks) {
await this.runSingle(task.id!);
}
}
}
}
+4 -1
View File
@@ -63,7 +63,10 @@ export default class CronViewService {
query: FindOptions<CrontabView>['where'],
): Promise<CrontabView> {
const doc: any = await CrontabViewModel.findOne({ where: { ...query } });
return doc && (doc.get({ plain: true }) as CrontabView);
if (!doc) {
throw new Error(`CronView ${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
public async disabled(ids: number[]) {
+41 -45
View File
@@ -3,12 +3,9 @@ import winston from 'winston';
import config from '../config';
import {
Dependence,
InstallDependenceCommandTypes,
DependenceStatus,
DependenceTypes,
unInstallDependenceCommandTypes,
DependenceModel,
GetDependenceCommandTypes,
versionDependenceCommandTypes,
} from '../data/dependence';
import { spawn } from 'cross-spawn';
@@ -19,6 +16,9 @@ import {
getPid,
killTask,
promiseExecSuccess,
getInstallCommand,
getUninstallCommand,
getGetCommand,
} from '../config/util';
import dayjs from 'dayjs';
import taskLimit from '../shared/pLimit';
@@ -28,7 +28,7 @@ export default class DependenceService {
constructor(
@Inject('logger') private logger: winston.Logger,
private sockService: SockService,
) {}
) { }
public async create(payloads: Dependence[]): Promise<Dependence[]> {
const tabs = payloads.map((x) => {
@@ -67,6 +67,9 @@ export default class DependenceService {
public async remove(ids: number[], force = false): Promise<Dependence[]> {
const docs = await DependenceModel.findAll({ where: { id: ids } });
for (const doc of docs) {
taskLimit.removeQueuedDependency(doc);
}
const unInstalledDeps = docs.filter(
(x) => x.status !== DependenceStatus.installed,
);
@@ -95,34 +98,32 @@ export default class DependenceService {
searchValue,
type,
status,
}: { searchValue: string; type: string; status: string },
}: {
searchValue: string;
type: keyof typeof DependenceTypes;
status: string;
},
sort: any = [],
query: any = {},
): Promise<Dependence[]> {
let condition = {
...query,
type: DependenceTypes[type as any],
};
let condition = query;
if (type && DependenceTypes[type] !== undefined) {
condition.type = DependenceTypes[type];
}
if (status) {
condition.status = status.split(',').map(Number);
}
if (searchValue) {
const encodeText = encodeURI(searchValue);
const reg = {
condition.name = {
[Op.or]: [
{ [Op.like]: `%${searchValue}%` },
{ [Op.like]: `%${encodeText}%` },
],
};
condition = {
...condition,
name: reg,
};
}
try {
const result = await this.find(condition, sort);
return result as any;
return await this.find(condition, sort);
} catch (error) {
throw error;
}
@@ -132,10 +133,12 @@ export default class DependenceService {
docs: Dependence[],
isInstall: boolean = true,
force: boolean = false,
) {
): Promise<void> {
docs.forEach((dep) => {
this.installOrUninstallDependency(dep, isInstall, force);
});
return taskLimit.waitDependencyQueueDone();
}
public async reInstall(ids: number[]): Promise<Dependence[]> {
@@ -145,6 +148,9 @@ export default class DependenceService {
);
const docs = await DependenceModel.findAll({ where: { id: ids } });
for (const doc of docs) {
taskLimit.removeQueuedDependency(doc);
}
this.installDependenceOneByOne(docs, true, true);
return docs;
}
@@ -153,13 +159,11 @@ export default class DependenceService {
const docs = await DependenceModel.findAll({ where: { id: ids } });
for (const doc of docs) {
taskLimit.removeQueuedDependency(doc);
const depInstallCommand = InstallDependenceCommandTypes[doc.type];
const depUnInstallCommand = unInstallDependenceCommandTypes[doc.type];
const installCmd = `${depInstallCommand} ${doc.name.trim()}`;
const unInstallCmd = `${depUnInstallCommand} ${doc.name.trim()}`;
const depInstallCommand = getInstallCommand(doc.type, doc.name);
const depUnInstallCommand = getUninstallCommand(doc.type, doc.name);
const pids = await Promise.all([
getPid(installCmd),
getPid(unInstallCmd),
getPid(depInstallCommand),
getPid(depUnInstallCommand),
]);
for (const pid of pids) {
pid && (await killTask(pid));
@@ -183,7 +187,10 @@ export default class DependenceService {
query: FindOptions<Dependence>['where'],
): Promise<Dependence> {
const doc: any = await DependenceModel.findOne({ where: { ...query } });
return doc && (doc.get({ plain: true }) as Dependence);
if (!doc) {
throw new Error(`Dependency ${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
private async updateLog(ids: number[], log: string): Promise<void> {
@@ -223,11 +230,9 @@ export default class DependenceService {
? 'installDependence'
: 'uninstallDependence';
let depName = dependency.name.trim();
const depRunCommand = (
isInstall
? InstallDependenceCommandTypes
: unInstallDependenceCommandTypes
)[dependency.type];
const command = isInstall
? getInstallCommand(dependency.type, depName)
: getUninstallCommand(dependency.type, depName);
const actionText = isInstall ? '安装' : '删除';
const startTime = dayjs();
@@ -243,7 +248,7 @@ export default class DependenceService {
// 判断是否已经安装过依赖
if (isInstall && !force) {
const getCommandPrefix = GetDependenceCommandTypes[dependency.type];
const getCommand = getGetCommand(dependency.type, depName);
const depVersionStr = versionDependenceCommandTypes[dependency.type];
let depVersion = '';
if (depName.includes(depVersionStr)) {
@@ -260,13 +265,7 @@ export default class DependenceService {
const isLinuxDependence = dependency.type === DependenceTypes.linux;
const isPythonDependence =
dependency.type === DependenceTypes.python3;
const depInfo = (
await promiseExecSuccess(
isNodeDependence
? `${getCommandPrefix} | grep "${depName}" | head -1`
: `${getCommandPrefix} ${depName}`,
)
)
const depInfo = (await promiseExecSuccess(getCommand))
.replace(/\s{2,}/, ' ')
.replace(/\s+$/, '');
@@ -301,12 +300,9 @@ export default class DependenceService {
const proxyStr = dependenceProxyFileExist
? `source ${config.dependenceProxyFile} &&`
: '';
const cp = spawn(
`${proxyStr} ${depRunCommand} ${dependency.name.trim()}`,
{
shell: '/bin/bash',
},
);
const cp = spawn(`${proxyStr} ${command}`, {
shell: '/bin/bash',
});
cp.stdout.on('data', async (data) => {
this.sockService.sendMessage({
@@ -350,7 +346,7 @@ export default class DependenceService {
});
this.updateLog(depIds, message);
let status = null;
let status: number;
if (isSucceed) {
status = isInstall
? DependenceStatus.installed
+43 -14
View File
@@ -1,7 +1,8 @@
import { Service, Inject } from 'typedi';
import groupBy from 'lodash/groupBy';
import { FindOptions, Op } from 'sequelize';
import { Inject, Service } from 'typedi';
import winston from 'winston';
import config from '../config';
import * as fs from 'fs/promises';
import {
Env,
EnvModel,
@@ -11,12 +12,12 @@ import {
minPosition,
stepPosition,
} from '../data/env';
import groupBy from 'lodash/groupBy';
import { FindOptions, Op } from 'sequelize';
import { writeFileWithLock } from '../shared/utils';
import { sequelize } from '../data';
@Service()
export default class EnvService {
constructor(@Inject('logger') private logger: winston.Logger) {}
constructor(@Inject('logger') private logger: winston.Logger) { }
public async create(payloads: Env[]): Promise<Env[]> {
const envs = await this.envs();
@@ -40,7 +41,7 @@ export default class EnvService {
}
public async insert(payloads: Env[]): Promise<Env[]> {
const result = [];
const result: Env[] = [];
for (const env of payloads) {
const doc = await EnvModel.create(env, { returning: true });
result.push(doc);
@@ -61,7 +62,7 @@ export default class EnvService {
return await this.getDb({ id: payload.id });
}
public async remove(ids: string[]) {
public async remove(ids: number[]) {
await EnvModel.destroy({ where: { id: ids } });
await this.set_envs();
}
@@ -146,10 +147,11 @@ export default class EnvService {
}
try {
const result = await this.find(condition, [
[sequelize.literal('COALESCE(`isPinned`, 0)'), 'DESC'],
['position', 'DESC'],
['createdAt', 'ASC'],
]);
return result as any;
return result;
} catch (error) {
throw error;
}
@@ -160,15 +162,18 @@ export default class EnvService {
where: { ...query },
order: [...sort],
});
return docs;
return docs.map((x) => x.get({ plain: true }));
}
public async getDb(query: FindOptions<Env>['where']): Promise<Env> {
const doc: any = await EnvModel.findOne({ where: { ...query } });
return doc && (doc.get({ plain: true }) as Env);
if (!doc) {
throw new Error(`Env ${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
public async disabled(ids: string[]) {
public async disabled(ids: number[]) {
await EnvModel.update(
{ status: EnvStatus.disabled },
{ where: { id: ids } },
@@ -176,16 +181,24 @@ export default class EnvService {
await this.set_envs();
}
public async enabled(ids: string[]) {
public async enabled(ids: number[]) {
await EnvModel.update({ status: EnvStatus.normal }, { where: { id: ids } });
await this.set_envs();
}
public async updateNames({ ids, name }: { ids: string[]; name: string }) {
public async updateNames({ ids, name }: { ids: number[]; name: string }) {
await EnvModel.update({ name }, { where: { id: ids } });
await this.set_envs();
}
public async pin(ids: number[]) {
await EnvModel.update({ isPinned: 1 }, { where: { id: ids } });
}
public async unPin(ids: number[]) {
await EnvModel.update({ isPinned: 0 }, { where: { id: ids } });
}
public async set_envs() {
const envs = await this.envs('', {
name: { [Op.not]: null },
@@ -193,6 +206,8 @@ export default class EnvService {
});
const groups = groupBy(envs, 'name');
let env_string = '';
let js_env_string = '';
let py_env_string = 'import os\n';
for (const key in groups) {
if (Object.prototype.hasOwnProperty.call(groups, key)) {
const group = groups[key];
@@ -205,9 +220,23 @@ export default class EnvService {
.replace(/'/g, "'\\''")
.trim();
env_string += `export ${key}='${value}'\n`;
const _env_value = `${group
.map((x) => x.value)
.join('&')
.replace(/\\/g, '\\\\')}`;
js_env_string += `process.env.${key}=\`${_env_value.replace(
/\`/g,
'\\`',
)}\`;\n`;
py_env_string += `os.environ['${key}']='''${_env_value.replace(
/\'/g,
"\\'",
)}'''\n`;
}
}
}
await fs.writeFile(config.envFile, env_string);
await writeFileWithLock(config.envFile, env_string);
await writeFileWithLock(config.jsEnvFile, js_env_string);
await writeFileWithLock(config.pyEnvFile, py_env_string);
}
}
+64
View File
@@ -0,0 +1,64 @@
import { Server, ServerCredentials } from '@grpc/grpc-js';
import { CronService } from '../protos/cron';
import { HealthService } from '../protos/health';
import { ApiService } from '../protos/api';
import { addCron } from '../schedule/addCron';
import { delCron } from '../schedule/delCron';
import { check } from '../schedule/health';
import * as Api from '../schedule/api';
import Logger from '../loaders/logger';
import { promisify } from 'util';
import config from '../config';
import { metricsService } from './metrics';
import { Service } from 'typedi';
@Service()
export class GrpcServerService {
private server: Server = new Server({ 'grpc.enable_http_proxy': 0 });
async initialize() {
try {
this.server.addService(HealthService, { check });
this.server.addService(CronService, { addCron, delCron });
this.server.addService(ApiService, Api);
const grpcPort = config.grpcPort;
const bindAsync = promisify(this.server.bindAsync).bind(this.server);
await bindAsync(
`0.0.0.0:${grpcPort}`,
ServerCredentials.createInsecure(),
);
Logger.debug(`✌️ gRPC service started successfully`);
metricsService.record('grpc_service_start', 1, {
port: grpcPort.toString(),
});
return grpcPort;
} catch (err) {
Logger.error('Failed to start gRPC service:', err);
throw err;
}
}
async shutdown() {
try {
if (this.server) {
await new Promise((resolve) => {
this.server.tryShutdown(() => {
Logger.debug('gRPC service stopped');
metricsService.record('grpc_service_stop', 1);
resolve(null);
});
});
}
} catch (err) {
Logger.error('Error while shutting down gRPC service:', err);
throw err;
}
}
getServer() {
return this.server;
}
}
+72
View File
@@ -0,0 +1,72 @@
import { Service } from 'typedi';
import Logger from '../loaders/logger';
import { GrpcServerService } from './grpc';
import { HttpServerService } from './http';
interface HealthStatus {
status: 'ok' | 'error';
services: {
http: boolean;
grpc: boolean;
};
metrics: {
uptime: number;
memory: {
used: number;
total: number;
};
};
}
@Service()
export class HealthService {
private startTime = Date.now();
constructor(
private grpcServerService: GrpcServerService,
private httpServerService: HttpServerService,
) {}
async check(): Promise<HealthStatus> {
const status: HealthStatus = {
status: 'ok',
services: {
http: true,
grpc: true,
},
metrics: {
uptime: Math.floor((Date.now() - this.startTime) / 1000),
memory: {
used: process.memoryUsage().heapUsed,
total: process.memoryUsage().heapTotal,
},
},
};
try {
const httpServer = this.httpServerService.getServer();
if (!httpServer) {
status.services.http = false;
status.status = 'error';
}
} catch (err) {
status.services.http = false;
status.status = 'error';
Logger.error('HTTP server check failed:', err);
}
try {
const grpcServer = this.grpcServerService.getServer();
if (!grpcServer) {
status.services.grpc = false;
status.status = 'error';
}
} catch (err) {
status.services.grpc = false;
status.status = 'error';
Logger.error('gRPC server check failed:', err);
}
return status;
}
}
+53
View File
@@ -0,0 +1,53 @@
import express from 'express';
import Logger from '../loaders/logger';
import { metricsService } from './metrics';
import { Service } from 'typedi';
import { Server } from 'http';
@Service()
export class HttpServerService {
private server?: Server = undefined;
async initialize(expressApp: express.Application, port: number) {
try {
return new Promise((resolve, reject) => {
this.server = expressApp.listen(port, '0.0.0.0', () => {
Logger.debug(`✌️ HTTP service started successfully`);
metricsService.record('http_service_start', 1, {
port: port.toString(),
});
resolve(this.server);
});
this.server?.on('error', (err: Error) => {
Logger.error('Failed to start HTTP service:', err);
reject(err);
});
});
} catch (err) {
Logger.error('Failed to start HTTP service:', err);
throw err;
}
}
async shutdown() {
try {
if (this.server) {
await new Promise((resolve) => {
this.server?.close(() => {
Logger.debug('HTTP service stopped');
metricsService.record('http_service_stop', 1);
resolve(null);
});
});
}
} catch (err) {
Logger.error('Error while shutting down HTTP service:', err);
throw err;
}
}
getServer() {
return this.server;
}
}
+14
View File
@@ -0,0 +1,14 @@
import path from 'path';
import { Inject, Service } from 'typedi';
import winston from 'winston';
import config from '../config';
@Service()
export default class LogService {
constructor(@Inject('logger') private logger: winston.Logger) {}
public checkFilePath(filePath: string, fileName: string) {
const finalPath = path.resolve(config.logPath, filePath, fileName);
return finalPath.startsWith(config.logPath) ? finalPath : '';
}
}
+92
View File
@@ -0,0 +1,92 @@
import { performance } from 'perf_hooks';
import Logger from '../loaders/logger';
interface Metric {
name: string;
value: number;
timestamp: number;
tags?: Record<string, string>;
}
class MetricsService {
private metrics: Metric[] = [];
private static instance: MetricsService;
private constructor() {
// 定期清理旧数据
setInterval(() => {
const oneHourAgo = Date.now() - 3600000;
this.metrics = this.metrics.filter(m => m.timestamp > oneHourAgo);
}, 60000);
}
static getInstance(): MetricsService {
if (!MetricsService.instance) {
MetricsService.instance = new MetricsService();
}
return MetricsService.instance;
}
record(name: string, value: number, tags?: Record<string, string>) {
this.metrics.push({
name,
value,
timestamp: Date.now(),
tags,
});
}
measure(name: string, fn: () => void, tags?: Record<string, string>) {
const start = performance.now();
try {
fn();
} finally {
const duration = performance.now() - start;
this.record(name, duration, tags);
}
}
async measureAsync(name: string, fn: () => Promise<void>, tags?: Record<string, string>) {
const start = performance.now();
try {
await fn();
} finally {
const duration = performance.now() - start;
this.record(name, duration, tags);
}
}
getMetrics(name?: string, tags?: Record<string, string>) {
let filtered = this.metrics;
if (name) {
filtered = filtered.filter(m => m.name === name);
}
if (tags) {
filtered = filtered.filter(m => {
if (!m.tags) return false;
return Object.entries(tags).every(([key, value]) => m.tags![key] === value);
});
}
return {
count: filtered.length,
average: filtered.reduce((acc, curr) => acc + curr.value, 0) / filtered.length,
min: Math.min(...filtered.map(m => m.value)),
max: Math.max(...filtered.map(m => m.value)),
metrics: filtered,
};
}
report() {
const report = {
timestamp: Date.now(),
metrics: this.getMetrics(),
};
Logger.info('性能指标报告:', report);
return report;
}
}
export const metricsService = MetricsService.getInstance();
+338 -195
View File
@@ -1,12 +1,11 @@
import crypto from 'crypto';
import got from 'got';
import { HttpProxyAgent, HttpsProxyAgent } from 'hpagent';
import nodemailer from 'nodemailer';
import { Inject, Service } from 'typedi';
import winston from 'winston';
import { parseBody, parseHeaders } from '../config/util';
import { NotificationInfo } from '../data/notify';
import UserService from './user';
import { httpClient } from '../config/http';
import { ProxyAgent } from 'undici';
@Service()
export default class NotificationService {
@@ -27,11 +26,14 @@ export default class NotificationService {
['aibotk', this.aibotk],
['iGot', this.iGot],
['pushPlus', this.pushPlus],
['wePlusBot', this.wePlusBot],
['email', this.email],
['pushMe', this.pushMe],
['webhook', this.webhook],
['lark', this.lark],
['chronocat', this.chronocat],
['ntfy', this.ntfy],
['wxPusherBot', this.wxPusherBot],
]);
private title = '';
@@ -42,22 +44,31 @@ export default class NotificationService {
retry: 1,
};
constructor(@Inject('logger') private logger: winston.Logger) {}
constructor() {}
public async notify(
title: string,
content: string,
notificationInfo?: NotificationInfo,
): Promise<boolean | undefined> {
const { type, ...rest } = await this.userService.getNotificationMode();
let { type, ...rest } = await this.userService.getNotificationMode();
if (notificationInfo?.type) {
type = notificationInfo?.type;
}
if (type) {
this.title = title;
this.content = content;
this.params = rest;
let params = rest;
if (notificationInfo) {
const { type: _, ...others } = notificationInfo;
params = { ...rest, ...others };
}
this.params = params;
const notificationModeAction = this.modeMap.get(type);
try {
return await notificationModeAction?.call(this);
} catch (error: any) {
throw error;
console.error(error);
}
}
return false;
@@ -82,8 +93,9 @@ export default class NotificationService {
private async gotify() {
const { gotifyUrl, gotifyToken, gotifyPriority = 1 } = this.params;
try {
const res: any = await got
.post(`${gotifyUrl}/message?token=${gotifyToken}`, {
const res = await httpClient.post(
`${gotifyUrl}/message?token=${gotifyToken}`,
{
...this.gotOption,
body: `title=${encodeURIComponent(
this.title,
@@ -93,8 +105,8 @@ export default class NotificationService {
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
})
.json();
},
);
if (typeof res.id === 'number') {
return true;
} else {
@@ -108,13 +120,11 @@ export default class NotificationService {
private async goCqHttpBot() {
const { goCqHttpBotQq, goCqHttpBotToken, goCqHttpBotUrl } = this.params;
try {
const res: any = await got
.post(`${goCqHttpBotUrl}?${goCqHttpBotQq}`, {
...this.gotOption,
json: { message: `${this.title}\n${this.content}` },
headers: { Authorization: 'Bearer ' + goCqHttpBotToken },
})
.json();
const res = await httpClient.post(`${goCqHttpBotUrl}?${goCqHttpBotQq}`, {
...this.gotOption,
json: { message: `${this.title}\n${this.content}` },
headers: { Authorization: 'Bearer ' + goCqHttpBotToken },
});
if (res.retcode === 0) {
return true;
} else {
@@ -127,17 +137,20 @@ export default class NotificationService {
private async serverChan() {
const { serverChanKey } = this.params;
const url = serverChanKey.startsWith('SCT')
? `https://sctapi.ftqq.com/${serverChanKey}.send`
: `https://sc.ftqq.com/${serverChanKey}.send`;
const matchResult = serverChanKey.match(/^sctp(\d+)t/i);
const url =
matchResult && matchResult[1]
? `https://${matchResult[1]}.push.ft07.com/send/${serverChanKey}.send`
: `https://sctapi.ftqq.com/${serverChanKey}.send`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `title=${this.title}&desp=${this.content}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `title=${encodeURIComponent(
this.title,
)}&desp=${encodeURIComponent(this.content)}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (res.errno === 0 || res.data.errno === 0) {
return true;
} else {
@@ -152,15 +165,13 @@ export default class NotificationService {
const { pushDeerKey, pushDeerUrl } = this.params;
const url = pushDeerUrl || `https://api2.pushdeer.com/message/push`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `pushkey=${pushDeerKey}&text=${encodeURIComponent(
this.title,
)}&desp=${encodeURIComponent(this.content)}&type=markdown`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `pushkey=${pushDeerKey}&text=${encodeURIComponent(
this.title,
)}&desp=${encodeURIComponent(this.content)}&type=markdown`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (
res.content.result.length !== undefined &&
res.content.result.length > 0
@@ -175,16 +186,13 @@ export default class NotificationService {
}
private async chat() {
const { chatUrl, chatToken } = this.params;
const url = `${chatUrl}${chatToken}`;
const { synologyChatUrl } = this.params;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `payload={"text":"${this.title}\n${this.content}"}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(synologyChatUrl, {
...this.gotOption,
body: `payload={"text":"${this.title}\n${this.content}"}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (res.success) {
return true;
} else {
@@ -196,24 +204,35 @@ export default class NotificationService {
}
private async bark() {
let { barkPush, barkIcon, barkSound, barkGroup, barkLevel, barkUrl } =
this.params;
let {
barkPush,
barkIcon = '',
barkSound = '',
barkGroup = '',
barkLevel = '',
barkUrl = '',
barkArchive = '',
} = this.params;
if (!barkPush.startsWith('http')) {
barkPush = `https://api.day.app/${barkPush}`;
}
const url = `${barkPush}/${encodeURIComponent(
this.title,
)}/${encodeURIComponent(
this.content,
)}?icon=${barkIcon}&sound=${barkSound}&group=${barkGroup}&level=${barkLevel}&url=${barkUrl}`;
const url = `${barkPush}`;
const body = {
title: this.title,
body: this.content,
icon: barkIcon,
sound: barkSound,
group: barkGroup,
isArchive: barkArchive,
level: barkLevel,
url: barkUrl,
};
try {
const res: any = await got
.get(url, {
...this.gotOption,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
json: body,
headers: { 'Content-Type': 'application/json' },
});
if (res.code === 200) {
return true;
} else {
@@ -239,29 +258,17 @@ export default class NotificationService {
}/bot${telegramBotToken}/sendMessage`;
let agent;
if (telegramBotProxyHost && telegramBotProxyPort) {
const options: any = {
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 256,
maxFreeSockets: 256,
proxy: `http://${authStr}${telegramBotProxyHost}:${telegramBotProxyPort}`,
};
const httpAgent = new HttpProxyAgent(options);
const httpsAgent = new HttpsProxyAgent(options);
agent = {
http: httpAgent,
https: httpsAgent,
};
agent = new ProxyAgent({
uri: `http://${authStr}${telegramBotProxyHost}:${telegramBotProxyPort}`,
});
}
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `chat_id=${telegramBotUserId}&text=${this.title}\n\n${this.content}&disable_web_page_preview=true`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
agent,
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `chat_id=${telegramBotUserId}&text=${this.title}\n\n${this.content}&disable_web_page_preview=true`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
dispatcher: agent,
});
if (res.ok) {
return true;
} else {
@@ -284,17 +291,15 @@ export default class NotificationService {
}
const url = `https://oapi.dingtalk.com/robot/send?access_token=${dingtalkBotToken}${secretParam}`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
const res = await httpClient.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
})
.json();
},
});
if (res.errcode === 0) {
return true;
} else {
@@ -310,17 +315,15 @@ export default class NotificationService {
this.params;
const url = `${weWorkOrigin}/cgi-bin/webhook/send?key=${weWorkBotKey}`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
const res = await httpClient.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
})
.json();
},
});
if (res.errcode === 0) {
return true;
} else {
@@ -337,15 +340,13 @@ export default class NotificationService {
const [corpid, corpsecret, touser, agentid, thumb_media_id = '1'] =
weWorkAppKey.split(',');
const url = `${weWorkOrigin}/cgi-bin/gettoken`;
const tokenRes: any = await got
.post(url, {
...this.gotOption,
json: {
corpid,
corpsecret,
},
})
.json();
const tokenRes = await httpClient.post(url, {
...this.gotOption,
json: {
corpid,
corpsecret,
},
});
let options: any = {
msgtype: 'mpnews',
@@ -386,20 +387,18 @@ export default class NotificationService {
}
try {
const res: any = await got
.post(
`${weWorkOrigin}/cgi-bin/message/send?access_token=${tokenRes.access_token}`,
{
...this.gotOption,
json: {
touser,
agentid,
safe: '0',
...options,
},
const res = await httpClient.post(
`${weWorkOrigin}/cgi-bin/message/send?access_token=${tokenRes.access_token}`,
{
...this.gotOption,
json: {
touser,
agentid,
safe: '0',
...options,
},
)
.json();
},
);
if (res.errcode === 0) {
return true;
@@ -441,14 +440,12 @@ export default class NotificationService {
}
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
...json,
},
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
json: {
...json,
},
});
if (res.code === 0) {
return true;
} else {
@@ -463,13 +460,11 @@ export default class NotificationService {
const { iGotPushKey } = this.params;
const url = `https://push.hellyw.com/${iGotPushKey.toLowerCase()}`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `title=${this.title}&content=${this.content}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `title=${this.title}&content=${this.content}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (res.ret === 0) {
return true;
@@ -482,20 +477,67 @@ export default class NotificationService {
}
private async pushPlus() {
const { pushPlusToken, pushPlusUser } = this.params;
const {
pushPlusToken,
pushPlusUser,
pushplusWebhook,
pushPlusTemplate,
pushplusChannel,
pushplusCallbackUrl,
pushplusTo,
} = this.params;
const url = `https://www.pushplus.plus/send`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
token: `${pushPlusToken}`,
title: `${this.title}`,
content: `${this.content.replace(/[\n\r]/g, '<br>')}`,
topic: `${pushPlusUser || ''}`,
},
})
.json();
let body = {
...this.gotOption,
json: {
token: `${pushPlusToken}`,
title: `${this.title}`,
content: `${this.content.replace(/[\n\r]/g, '<br>')}`,
topic: `${pushPlusUser || ''}`,
template: `${pushPlusTemplate || 'html'}`,
channel: `${pushplusChannel || 'wechat'}`,
webhook: `${pushplusWebhook || ''}`,
callbackUrl: `${pushplusCallbackUrl || ''}`,
to: `${pushplusTo || ''}`,
},
};
const res = await httpClient.post(url, body);
if (res.code === 200) {
return true;
} else {
throw new Error(JSON.stringify(res));
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
}
}
private async wePlusBot() {
const { wePlusBotToken, wePlusBotReceiver, wePlusBotVersion } = this.params;
let content = this.content;
let template = 'txt';
if (this.content.length > 800) {
template = 'html';
content = content.replace(/[\n\r]/g, '<br>');
}
const url = `https://www.weplusbot.com/send`;
try {
const res = await httpClient.post(url, {
...this.gotOption,
json: {
token: `${wePlusBotToken}`,
title: `${this.title}`,
template: `${template}`,
content: `${content}`,
receiver: `${wePlusBotReceiver || ''}`,
version: `${wePlusBotVersion || 'pro'}`,
},
});
if (res.code === 200) {
return true;
@@ -508,24 +550,36 @@ export default class NotificationService {
}
private async lark() {
let { larkKey } = this.params;
let { larkKey, larkSecret } = this.params;
if (!larkKey.startsWith('http')) {
larkKey = `https://open.feishu.cn/open-apis/bot/v2/hook/${larkKey}`;
}
const body: Record<string, any> = {
msg_type: 'text',
content: { text: `${this.title}\n\n${this.content}` },
};
// Add signature if secret is provided
// Note: Feishu's signature algorithm uses timestamp+"\n"+secret as the HMAC key
// and signs an empty message, which differs from typical HMAC usage
if (larkSecret) {
const timestamp = Math.floor(Date.now() / 1000).toString();
const stringToSign = `${timestamp}\n${larkSecret}`;
const hmac = crypto.createHmac('sha256', stringToSign);
const sign = hmac.digest('base64');
body.timestamp = timestamp;
body.sign = sign;
}
try {
const res: any = await got
.post(larkKey, {
...this.gotOption,
json: {
msg_type: 'text',
content: { text: `${this.title}\n\n${this.content}` },
},
headers: { 'Content-Type': 'application/json' },
})
.json();
if (res.StatusCode === 0) {
const res = await httpClient.post(larkKey, {
...this.gotOption,
json: body,
headers: { 'Content-Type': 'application/json' },
});
if (res.StatusCode === 0 || res.code === 0) {
return true;
} else {
throw new Error(JSON.stringify(res));
@@ -536,7 +590,7 @@ export default class NotificationService {
}
private async email() {
const { emailPass, emailService, emailUser } = this.params;
const { emailPass, emailService, emailUser, emailTo } = this.params;
try {
const transporter = nodemailer.createTransport({
@@ -549,7 +603,7 @@ export default class NotificationService {
const info = await transporter.sendMail({
from: `"青龙快讯" <${emailUser}>`,
to: `${emailUser}`,
to: emailTo ? emailTo.split(';') : emailUser,
subject: `${this.title}`,
html: `${this.content.replace(/\n/g, '<br/>')}`,
});
@@ -567,23 +621,124 @@ export default class NotificationService {
}
private async pushMe() {
const { pushMeKey } = this.params;
const { pushMeKey, pushMeUrl } = this.params;
try {
const res: any = await got.post(
`https://push.i-i.me/?push_key=${pushMeKey}`,
const res = await httpClient.post<'text'>(
pushMeUrl || 'https://push.i-i.me/',
{
...this.gotOption,
json: {
push_key: pushMeKey,
title: this.title,
content: this.content,
},
headers: { 'Content-Type': 'application/json' },
},
);
if (res.body === 'success') {
if (res === 'success') {
return true;
} else {
throw new Error(res.body);
throw new Error(res);
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
}
}
private async ntfy() {
const {
ntfyUrl,
ntfyTopic,
ntfyPriority,
ntfyToken,
ntfyUsername,
ntfyPassword,
ntfyActions,
} = this.params;
// 编码函数
const encodeRfc2047 = (text: string, charset: string = 'UTF-8'): string => {
const encodedText = Buffer.from(text).toString('base64');
return `=?${charset}?B?${encodedText}?=`;
};
try {
const headers: Record<string, string> = {
Title: encodeRfc2047(this.title),
Priority: `${ntfyPriority || '3'}`,
Icon: 'https://qn.whyour.cn/logo.png',
};
if (ntfyToken) {
headers['Authorization'] = `Bearer ${ntfyToken}`;
} else if (ntfyUsername && ntfyPassword) {
headers['Authorization'] = `Basic ${Buffer.from(
`${ntfyUsername}:${ntfyPassword}`,
).toString('base64')}`;
}
if (ntfyActions) {
headers['Actions'] = encodeRfc2047(ntfyActions);
}
const res = await httpClient.request(
`${ntfyUrl || 'https://ntfy.sh'}/${ntfyTopic}`,
{
...this.gotOption,
body: `${this.content}`,
headers: headers,
method: 'POST',
},
);
if (res.statusCode === 200) {
return true;
} else {
throw new Error(await res.body.text());
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
}
}
private async wxPusherBot() {
const { wxPusherBotAppToken, wxPusherBotTopicIds, wxPusherBotUids } =
this.params;
// 处理 topicIds,将分号分隔的字符串转为数组
const topicIds = wxPusherBotTopicIds
? wxPusherBotTopicIds
.split(';')
.map((id) => id.trim())
.filter((id) => id)
.map((id) => parseInt(id))
: [];
// 处理 uids,将分号分隔的字符串转为数组
const uids = wxPusherBotUids
? wxPusherBotUids
.split(';')
.map((uid) => uid.trim())
.filter((uid) => uid)
: [];
// topic_ids 和 uids 至少要有一个
if (!topicIds.length && !uids.length) {
throw new Error('wxPusher 服务的 TopicIds 和 Uids 至少配置一个才行');
}
const url = `https://wxpusher.zjiecode.com/api/send/message`;
try {
const res = await httpClient.post(url, {
...this.gotOption,
json: {
appToken: wxPusherBotAppToken,
content: `<h1>${this.title}</h1><br/><div style='white-space: pre-wrap;'>${this.content}</div>`,
summary: this.title,
contentType: 2,
topicIds: topicIds,
uids: uids,
verifyPayType: 0,
},
});
if (res.code === 1000) {
return true;
} else {
throw new Error(JSON.stringify(res));
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
@@ -629,15 +784,16 @@ export default class NotificationService {
},
],
};
const res: any = await got.post(url, {
const res = await httpClient.request(url, {
...this.gotOption,
json: data,
headers,
method: 'POST',
});
if (res.statusCode === 200) {
return true;
} else {
throw new Error(res.body);
throw new Error(await res.body.text());
}
}
}
@@ -656,17 +812,14 @@ export default class NotificationService {
webhookContentType,
} = this.params;
const { formatBody, formatUrl } = this.formatNotifyContent(
webhookUrl,
webhookBody,
);
if (!formatUrl && !formatBody) {
if (!webhookUrl?.includes('$title') && !webhookBody?.includes('$title')) {
throw new Error('Url 或者 Body 中必须包含 $title');
}
const headers = parseHeaders(webhookHeaders);
const body = parseBody(formatBody, webhookContentType);
const body = parseBody(webhookBody, webhookContentType, (v) =>
v?.replaceAll('$title', this.title)?.replaceAll('$content', this.content),
);
const bodyParam = this.formatBody(webhookContentType, body);
const options = {
method: webhookMethod,
@@ -675,12 +828,17 @@ export default class NotificationService {
allowGetBody: true,
...bodyParam,
};
try {
const res = await got(formatUrl, options);
const formatUrl = webhookUrl
?.replaceAll('$title', encodeURIComponent(this.title))
?.replaceAll('$content', encodeURIComponent(this.content));
const res = await httpClient.request(formatUrl, options);
const text = await res.body.text();
if (String(res.statusCode).startsWith('20')) {
return true;
} else {
throw new Error(JSON.stringify(res));
throw new Error(await res.body.text());
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
@@ -700,19 +858,4 @@ export default class NotificationService {
}
return {};
}
private formatNotifyContent(url: string, body: string) {
if (!url.includes('$title') && !body.includes('$title')) {
return {};
}
return {
formatUrl: url
?.replaceAll('$title', encodeURIComponent(this.title))
?.replaceAll('$content', encodeURIComponent(this.content)),
formatBody: body
?.replaceAll('$title', this.title)
?.replaceAll('$content', this.content),
};
}
}
+33 -22
View File
@@ -1,19 +1,18 @@
import { Service, Inject } from 'typedi';
import winston from 'winston';
import { createRandomString } from '../config/util';
import config from '../config';
import { App, AppModel } from '../data/open';
import { v4 as uuidV4 } from 'uuid';
import sequelize, { Op } from 'sequelize';
import { shareStore } from '../shared/store';
@Service()
export default class OpenService {
constructor(@Inject('logger') private logger: winston.Logger) {}
public async findTokenByValue(token: string): Promise<App | null> {
public async findApps(): Promise<App[] | null> {
const docs = await this.find({});
const doc = docs.filter((x) => x.tokens?.find((y) => y.value === token));
return doc[0];
return docs;
}
public async create(payload: App): Promise<App> {
@@ -21,12 +20,14 @@ export default class OpenService {
tab.client_id = createRandomString(12, 12);
tab.client_secret = createRandomString(24, 24);
const doc = await this.insert(tab);
const apps = await this.find({});
await shareStore.updateApps(apps);
return { ...doc, tokens: [] };
}
public async insert(payload: App): Promise<App> {
const doc = await AppModel.create(payload, { returning: true });
return doc.get({ plain: true }) as App;
return doc.get({ plain: true });
}
public async update(payload: App): Promise<App> {
@@ -34,26 +35,33 @@ export default class OpenService {
name: payload.name,
scopes: payload.scopes,
id: payload.id,
} as any);
} as App);
return { ...newDoc, tokens: [] };
}
private async updateDb(payload: App): Promise<App> {
private async updateDb(payload: Partial<App>): Promise<App> {
await AppModel.update(payload, { where: { id: payload.id } });
return await this.getDb({ id: payload.id });
const apps = await this.find({});
await shareStore.updateApps(apps);
return apps?.find((x) => x.id === payload.id) as App;
}
public async getDb(query: any): Promise<App> {
const doc: any = await AppModel.findOne({ where: query });
return doc && (doc.get({ plain: true }) as App);
public async getDb(query: Record<string, any>): Promise<App> {
const doc = await AppModel.findOne({ where: query });
if (!doc) {
throw new Error(`App ${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
public async remove(ids: number[]) {
await AppModel.destroy({ where: { id: ids } });
const apps = await this.find({});
await shareStore.updateApps(apps);
}
public async resetSecret(id: number): Promise<App> {
const tab: any = {
const tab: Partial<App> = {
client_secret: createRandomString(24, 24),
tokens: [],
id,
@@ -71,7 +79,7 @@ export default class OpenService {
public async list(
searchText: string = '',
sort: any = {},
query: any = {},
query: Record<string, any> = {},
): Promise<App[]> {
let condition = { ...query };
if (searchText) {
@@ -98,7 +106,7 @@ export default class OpenService {
}
}
private async find(query: any, sort?: any): Promise<App[]> {
private async find(query: Record<string, any>, sort?: any): Promise<App[]> {
const docs = await AppModel.findAll({ where: { ...query } });
return docs.map((x) => x.get({ plain: true }));
}
@@ -132,6 +140,8 @@ export default class OpenService {
{ tokens },
{ where: { client_id, client_secret } },
);
const apps = await this.find({});
await shareStore.updateApps(apps);
return {
code: 200,
data: {
@@ -141,7 +151,7 @@ export default class OpenService {
},
};
} else {
return { code: 400, message: 'client_idclient_seret有误' };
return { code: 400, message: 'client_idclient_seret 有误' };
}
}
@@ -149,14 +159,15 @@ export default class OpenService {
value: string;
expiration: number;
}> {
let systemApp = (await AppModel.findOne({
where: { name: 'system' },
})) as App;
const apps = await shareStore.getApps();
const systemApp = apps?.find((x) => x.name === 'system');
if (!systemApp) {
systemApp = await this.create({
name: 'system',
scopes: ['crons', 'system'],
} as App);
throw new Error('system app not found');
}
const now = Math.round(Date.now() / 1000);
const currentToken = systemApp.tokens?.find((x) => x.expiration > now);
if (currentToken) {
return currentToken;
}
const { data } = await this.authToken({
client_id: systemApp.client_id,
+48 -14
View File
@@ -13,10 +13,11 @@ import taskLimit from '../shared/pLimit';
import { spawn } from 'cross-spawn';
export interface ScheduleTaskType {
id: number;
id?: number;
command: string;
name?: string;
schedule?: string;
runOrigin: 'subscription' | 'system' | 'script';
}
export interface TaskCallbacks {
@@ -40,9 +41,13 @@ export default class ScheduleService {
private intervalSchedule = new ToadScheduler();
private maxBuffer = 200 * 1024 * 1024;
private taskLimitMap = {
system: 'runWithSystemLimit' as const,
script: 'runWithScriptLimit' as const,
subscription: 'runWithSubscriptionLimit' as const,
};
constructor(@Inject('logger') private logger: winston.Logger) { }
constructor(@Inject('logger') private logger: winston.Logger) {}
async runTask(
command: string,
@@ -51,12 +56,21 @@ export default class ScheduleService {
schedule?: string;
name?: string;
command?: string;
id: string;
runOrigin: 'subscription' | 'system' | 'script';
},
completionTime: 'start' | 'end' = 'end',
) {
return taskLimit.runWithCronLimit(() => {
const { runOrigin, ...others } = params;
return taskLimit[this.taskLimitMap[runOrigin]](others, () => {
return new Promise(async (resolve, reject) => {
this.logger.info(`[panel][开始执行任务] 参数 ${JSON.stringify({ ...params, command })}`);
this.logger.info(
`[panel][开始执行任务] 参数: ${JSON.stringify({
...others,
command,
})}`,
);
try {
const startTime = dayjs();
@@ -90,13 +104,21 @@ export default class ScheduleService {
});
cp.on('exit', async (code) => {
this.logger.info(
'[panel][执行任务结束] 参数: %s, 退出码: %j',
JSON.stringify({
...others,
command,
}),
code,
);
const endTime = dayjs();
await callbacks.onEnd?.(
cp,
endTime,
endTime.diff(startTime, 'seconds'),
);
resolve({ ...params, pid: cp.pid, code });
resolve({ ...others, pid: cp.pid, code });
});
} catch (error) {
this.logger.error(
@@ -111,13 +133,13 @@ export default class ScheduleService {
}
async createCronTask(
{ id = 0, command, name, schedule = '' }: ScheduleTaskType,
{ id = 0, command, name, schedule = '', runOrigin }: ScheduleTaskType,
callbacks?: TaskCallbacks,
runImmediately = false,
) {
const _id = this.formatId(id);
this.logger.info(
'[panel][创建cron任务], 任务ID: %s, cron: %s, 任务名: %s, 执行命令: %s',
'[panel][创建cron任务] 任务ID: %s, cron: %s, 任务名: %s, 执行命令: %s',
_id,
schedule,
name,
@@ -131,6 +153,8 @@ export default class ScheduleService {
name,
schedule,
command,
id: _id,
runOrigin,
});
}),
);
@@ -140,13 +164,15 @@ export default class ScheduleService {
name,
schedule,
command,
id: _id,
runOrigin,
});
}
}
async cancelCronTask({ id = 0, name }: ScheduleTaskType) {
const _id = this.formatId(id);
this.logger.info('[panel][取消定时任务], 任务名: %s', name);
this.logger.info('[panel][取消定时任务] 任务名: %s', name);
if (this.scheduleStacks.has(_id)) {
this.scheduleStacks.get(_id)?.cancel();
this.scheduleStacks.delete(_id);
@@ -154,14 +180,14 @@ export default class ScheduleService {
}
async createIntervalTask(
{ id = 0, command, name = '' }: ScheduleTaskType,
{ id = 0, command, name = '', runOrigin }: ScheduleTaskType,
schedule: SimpleIntervalSchedule,
runImmediately = true,
callbacks?: TaskCallbacks,
) {
const _id = this.formatId(id);
this.logger.info(
'[panel][创建interval任务], 任务ID: %s, 任务名: %s, 执行命令: %s',
'[panel][创建interval任务] 任务ID: %s, 任务名: %s, 执行命令: %s',
_id,
name,
command,
@@ -172,11 +198,13 @@ export default class ScheduleService {
this.runTask(command, callbacks, {
name,
command,
id: _id,
runOrigin,
});
},
(err) => {
this.logger.error(
'[执行任务失败] 命令: %s, 错误信息: %j',
'[panel][执行任务失败] 命令: %s, 错误信息: %j',
command,
err,
);
@@ -186,7 +214,7 @@ export default class ScheduleService {
const job = new LongIntervalJob(
{ runImmediately: false, ...schedule },
task,
_id,
{ id: _id },
);
this.intervalSchedule.addIntervalJob(job);
@@ -195,13 +223,19 @@ export default class ScheduleService {
this.runTask(command, callbacks, {
name,
command,
id: _id,
runOrigin,
});
}
}
async cancelIntervalTask({ id = 0, name }: ScheduleTaskType) {
const _id = this.formatId(id);
this.logger.info('[取消interval任务], 任务ID: %s, 任务名: %s', _id, name);
this.logger.info(
'[panel][取消interval任务] 任务ID: %s, 任务名: %s',
_id,
name,
);
this.intervalSchedule.removeById(_id);
}
+16 -4
View File
@@ -7,6 +7,7 @@ import ScheduleService, { TaskCallbacks } from './schedule';
import config from '../config';
import { TASK_COMMAND } from '../config/const';
import { getFileContentByName, getPid, killTask, rmPath } from '../config/util';
import taskLimit from '../shared/pLimit';
@Service()
export default class ScriptService {
@@ -41,9 +42,9 @@ export default class ScriptService {
const relativePath = path.relative(config.scriptPath, filePath);
const command = `${TASK_COMMAND} ${relativePath} now`;
const pid = await this.scheduleService.runTask(
command,
`real_time=true ${command}`,
this.taskCallbacks(filePath),
{ command },
{ command, id: relativePath.replace(/ /g, '-'), runOrigin: 'script' },
'start',
);
@@ -53,6 +54,7 @@ export default class ScriptService {
public async stopScript(filePath: string, pid: number) {
if (!pid) {
const relativePath = path.relative(config.scriptPath, filePath);
taskLimit.removeQueuedCron(relativePath.replace(/ /g, '-'));
pid = (await getPid(`${TASK_COMMAND} ${relativePath} now`)) as number;
}
try {
@@ -62,9 +64,19 @@ export default class ScriptService {
return { code: 200 };
}
public checkFilePath(filePath: string, fileName: string) {
const finalPath = path.resolve(config.scriptPath, filePath, fileName);
return finalPath.startsWith(config.scriptPath) ? finalPath : '';
}
public async getFile(filePath: string, fileName: string) {
const _filePath = join(config.scriptPath, filePath, fileName);
const content = await getFileContentByName(_filePath);
const finalPath = this.checkFilePath(filePath, fileName);
if (!finalPath) {
return '';
}
const content = await getFileContentByName(finalPath);
return content;
}
}
+46 -9
View File
@@ -7,6 +7,7 @@ import { Subscription } from '../data/subscription';
import { formatUrl } from '../config/subscription';
import config from '../config';
import { fileExist, rmPath } from '../config/util';
import { writeFileWithLock } from '../shared/utils';
@Service()
export default class SshKeyService {
@@ -25,13 +26,13 @@ export default class SshKeyService {
if (_exist) {
config = await fs.readFile(this.sshConfigFilePath, { encoding: 'utf-8' });
} else {
await fs.writeFile(this.sshConfigFilePath, '');
await writeFileWithLock(this.sshConfigFilePath, '', { mode: '600' });
}
if (!config.includes(this.sshConfigHeader)) {
await fs.writeFile(
await writeFileWithLock(
this.sshConfigFilePath,
`${this.sshConfigHeader}\n\n${config}`,
{ encoding: 'utf-8' },
{ mode: '600' },
);
}
}
@@ -41,10 +42,13 @@ export default class SshKeyService {
key: string,
): Promise<void> {
try {
await fs.writeFile(path.join(this.sshPath, alias), `${key}${os.EOL}`, {
encoding: 'utf8',
mode: '400',
});
await writeFileWithLock(
path.join(this.sshPath, alias),
`${key}${os.EOL}`,
{
mode: '400',
},
);
} catch (error) {
this.logger.error('生成私钥文件失败', error);
}
@@ -74,11 +78,12 @@ export default class SshKeyService {
this.sshPath,
alias,
)}\n StrictHostKeyChecking no\n${proxyStr}`;
await fs.writeFile(
await writeFileWithLock(
`${path.join(this.sshPath, `${alias}.config`)}`,
config,
{
encoding: 'utf8',
mode: '600',
},
);
}
@@ -102,7 +107,11 @@ export default class SshKeyService {
await this.generateSingleSshConfig(alias, host, proxy);
}
public async removeSSHKey(alias: string, host: string, proxy?: string): Promise<void> {
public async removeSSHKey(
alias: string,
host: string,
proxy?: string,
): Promise<void> {
await this.removePrivateKeyFile(alias);
await this.removeSshConfig(alias);
}
@@ -122,4 +131,32 @@ export default class SshKeyService {
}
}
}
public async addGlobalSSHKey(key: string, alias: string): Promise<void> {
await this.generatePrivateKeyFile(`~global_${alias}`, key);
// Create a global SSH config entry that matches all hosts
// This allows the key to be used for any Git repository
await this.generateGlobalSshConfig(`~global_${alias}`);
}
public async removeGlobalSSHKey(alias: string): Promise<void> {
await this.removePrivateKeyFile(`~global_${alias}`);
await this.removeSshConfig(`~global_${alias}`);
}
private async generateGlobalSshConfig(alias: string) {
// Create a config that matches all hosts, making this key globally available
const config = `Host *\n IdentityFile ${path.join(
this.sshPath,
alias,
)}\n StrictHostKeyChecking no\n`;
await writeFileWithLock(
`${path.join(this.sshPath, `${alias}.config`)}`,
config,
{
encoding: 'utf8',
mode: '600',
},
);
}
}
+39 -22
View File
@@ -3,6 +3,7 @@ import winston from 'winston';
import config from '../config';
import {
Subscription,
SubscriptionInstance,
SubscriptionModel,
SubscriptionStatus,
} from '../data/subscription';
@@ -29,6 +30,8 @@ import { LOG_END_SYMBOL } from '../config/const';
import { formatCommand, formatUrl } from '../config/subscription';
import { CrontabModel } from '../data/cron';
import CrontabService from './cron';
import taskLimit from '../shared/pLimit';
import { logStreamManager } from '../shared/logStreamManager';
@Service()
export default class SubscriptionService {
@@ -40,8 +43,12 @@ export default class SubscriptionService {
private crontabService: CrontabService,
) {}
public async list(searchText?: string): Promise<Subscription[]> {
public async list(
searchText?: string,
ids?: string,
): Promise<SubscriptionInstance[]> {
let query = {};
const subIds = JSON.parse(ids || '[]');
if (searchText) {
const reg = {
[Op.or]: [
@@ -62,7 +69,7 @@ export default class SubscriptionService {
}
try {
const result = await SubscriptionModel.findAll({
where: query,
where: { ...query, ...(ids ? { id: subIds } : undefined) },
order: [
['is_disabled', 'ASC'],
['createdAt', 'DESC'],
@@ -87,16 +94,16 @@ export default class SubscriptionService {
this.scheduleService.cancelCronTask(doc as any);
needCreate &&
(await this.scheduleService.createCronTask(
doc as any,
{ ...doc, runOrigin: 'subscription' } as any,
this.taskCallbacks(doc),
runImmediately,
));
} else {
} else if (doc.interval_schedule) {
this.scheduleService.cancelIntervalTask(doc as any);
const { type, value } = doc.interval_schedule as any;
const { type, value } = doc.interval_schedule;
needCreate &&
(await this.scheduleService.createIntervalTask(
doc as any,
{ ...doc, runOrigin: 'subscription' } as any,
{ [type]: value } as SimpleIntervalSchedule,
runImmediately,
this.taskCallbacks(doc),
@@ -130,7 +137,7 @@ export default class SubscriptionService {
let beforeStr = '';
try {
if (doc.sub_before) {
await fs.appendFile(absolutePath, `\n## 执行before命令...\n\n`);
await logStreamManager.write(absolutePath, `\n## 执行before命令...\n\n`);
beforeStr = await promiseExec(doc.sub_before);
}
} catch (error: any) {
@@ -138,7 +145,7 @@ export default class SubscriptionService {
(error.stderr && error.stderr.toString()) || JSON.stringify(error);
}
if (beforeStr) {
await fs.appendFile(absolutePath, `${beforeStr}\n`);
await logStreamManager.write(absolutePath, `${beforeStr}\n`);
}
},
onStart: async (cp: ChildProcessWithoutNullStreams, startTime) => {
@@ -157,7 +164,7 @@ export default class SubscriptionService {
let afterStr = '';
try {
if (sub.sub_after) {
await fs.appendFile(absolutePath, `\n\n## 执行after命令...\n\n`);
await logStreamManager.write(absolutePath, `\n\n## 执行after命令...\n\n`);
afterStr = await promiseExec(sub.sub_after);
}
} catch (error: any) {
@@ -165,16 +172,19 @@ export default class SubscriptionService {
(error.stderr && error.stderr.toString()) || JSON.stringify(error);
}
if (afterStr) {
await fs.appendFile(absolutePath, `${afterStr}\n`);
await logStreamManager.write(absolutePath, `${afterStr}\n`);
}
await fs.appendFile(
await logStreamManager.write(
absolutePath,
`\n## 执行结束... ${endTime.format(
'YYYY-MM-DD HH:mm:ss',
)} ${diff} ${LOG_END_SYMBOL}`,
);
// Close the stream after task completion
await logStreamManager.closeStream(absolutePath);
await SubscriptionModel.update(
{ status: SubscriptionStatus.idle, pid: undefined },
{ where: { id: sub.id } },
@@ -189,12 +199,12 @@ export default class SubscriptionService {
onError: async (message: string) => {
const sub = await this.getDb({ id: doc.id });
const absolutePath = await handleLogPath(sub.log_path as string);
await fs.appendFile(absolutePath, `\n${message}`);
await logStreamManager.write(absolutePath, `\n${message}`);
},
onLog: async (message: string) => {
const sub = await this.getDb({ id: doc.id });
const absolutePath = await handleLogPath(sub.log_path as string);
await fs.appendFile(absolutePath, `\n${message}`);
await logStreamManager.write(absolutePath, `\n${message}`);
},
};
}
@@ -202,12 +212,12 @@ export default class SubscriptionService {
public async create(payload: Subscription): Promise<Subscription> {
const tab = new Subscription(payload);
const doc = await this.insert(tab);
await this.handleTask(doc);
await this.handleTask(doc.get({ plain: true }));
await this.setSshConfig();
return doc;
}
public async insert(payload: Subscription): Promise<Subscription> {
public async insert(payload: Subscription): Promise<SubscriptionInstance> {
return await SubscriptionModel.create(payload, { returning: true });
}
@@ -256,10 +266,10 @@ export default class SubscriptionService {
);
}
public async remove(ids: number[], query: any) {
public async remove(ids: number[], query: { force?: boolean }) {
const docs = await SubscriptionModel.findAll({ where: { id: ids } });
for (const doc of docs) {
await this.handleTask(doc, false);
await this.handleTask(doc.get({ plain: true }), false);
}
await SubscriptionModel.destroy({ where: { id: ids } });
await this.setSshConfig();
@@ -271,7 +281,9 @@ export default class SubscriptionService {
}
for (const doc of docs) {
const filePath = join(config.scriptPath, doc.alias);
const repoPath = join(config.repoPath, doc.alias);
await rmPath(filePath);
await rmPath(repoPath);
}
}
}
@@ -279,8 +291,11 @@ export default class SubscriptionService {
public async getDb(
query: FindOptions<Subscription>['where'],
): Promise<Subscription> {
const doc: any = await SubscriptionModel.findOne({ where: { ...query } });
return doc && (doc.get({ plain: true }) as Subscription);
const doc = await SubscriptionModel.findOne({ where: { ...query } });
if (!doc) {
throw new Error(`Subscription ${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
public async run(ids: number[]) {
@@ -323,6 +338,8 @@ export default class SubscriptionService {
name: subscription.name,
schedule: subscription.schedule,
command,
id: String(subscription.id),
runOrigin: 'subscription',
});
}
@@ -331,7 +348,7 @@ export default class SubscriptionService {
const docs = await SubscriptionModel.findAll({ where: { id: ids } });
await this.setSshConfig();
for (const doc of docs) {
await this.handleTask(doc, false);
await this.handleTask(doc.get({ plain: true }), false);
}
}
@@ -340,7 +357,7 @@ export default class SubscriptionService {
const docs = await SubscriptionModel.findAll({ where: { id: ids } });
await this.setSshConfig();
for (const doc of docs) {
await this.handleTask(doc);
await this.handleTask(doc.get({ plain: true }));
}
}
@@ -371,7 +388,7 @@ export default class SubscriptionService {
files.map(async (x) => ({
filename: x,
directory: relativeDir.replace(config.logPath, ''),
time: (await fs.stat(`${dir}/${x}`)).mtime.getTime(),
time: (await fs.lstat(`${dir}/${x}`)).birthtimeMs,
})),
)
).sort((a, b) => b.time - a.time);
+194 -80
View File
@@ -1,20 +1,13 @@
import { spawn } from 'cross-spawn';
import { Response } from 'express';
import { Service, Inject } from 'typedi';
import fs from 'fs';
import { Agent, request } from 'undici';
import sum from 'lodash/sum';
import path from 'path';
import { Inject, Service } from 'typedi';
import winston from 'winston';
import config from '../config';
import {
AuthDataType,
AuthInfo,
SystemInstance,
SystemModel,
SystemModelInfo,
} from '../data/system';
import { NotificationInfo } from '../data/notify';
import NotificationService from './notify';
import ScheduleService, { TaskCallbacks } from './schedule';
import { spawn } from 'cross-spawn';
import SockService from './sock';
import got from 'got';
import { NotificationModeStringMap, TASK_COMMAND } from '../config/const';
import {
getPid,
killTask,
@@ -22,14 +15,28 @@ import {
parseVersion,
promiseExec,
readDirs,
rmPath,
setSystemTimezone,
} from '../config/util';
import { TASK_COMMAND } from '../config/const';
import {
DependenceModel,
DependenceStatus,
DependenceTypes,
} from '../data/dependence';
import { NotificationInfo } from '../data/notify';
import {
AuthDataType,
SystemInfo,
SystemInstance,
SystemModel,
SystemModelInfo,
} from '../data/system';
import taskLimit from '../shared/pLimit';
import tar from 'tar';
import path from 'path';
import fs from 'fs';
import sum from 'lodash/sum';
import { DependenceModel, DependenceStatus, DependenceTypes } from '../data/dependence';
import NotificationService from './notify';
import ScheduleService, { TaskCallbacks } from './schedule';
import SockService from './sock';
import os from 'os';
import dayjs from 'dayjs';
@Service()
export default class SystemService {
@@ -40,22 +47,29 @@ export default class SystemService {
@Inject('logger') private logger: winston.Logger,
private scheduleService: ScheduleService,
private sockService: SockService,
) {}
) { }
public async getSystemConfig() {
const doc = await this.getDb({ type: AuthDataType.systemConfig });
return doc || ({} as SystemInstance);
return {
...doc,
info: { ...doc.info, timezone: doc.info?.timezone || 'Asia/Shanghai' },
};
}
private async updateAuthDb(payload: AuthInfo): Promise<SystemInstance> {
await SystemModel.upsert({ ...payload });
const doc = await this.getDb({ type: payload.type });
private async updateAuthDb(payload: SystemInfo): Promise<SystemInfo> {
const { id, ...others } = payload;
await SystemModel.update(others, { where: { id } });
const doc = await this.getDb({ id });
return doc;
}
public async getDb(query: any): Promise<SystemInstance> {
const doc: any = await SystemModel.findOne({ where: { ...query } });
return doc && doc.get({ plain: true });
public async getDb(query: any): Promise<SystemInfo> {
const doc = await SystemModel.findOne({ where: query });
if (!doc) {
throw new Error(`System ${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
public async updateNotificationMode(notificationInfo: NotificationInfo) {
@@ -83,17 +97,22 @@ export default class SystemService {
info: { ...oDoc.info, ...info },
});
const cron = {
id: result.id || NaN,
id: result.id as number,
name: '删除日志',
command: `ql rmlog ${info.logRemoveFrequency}`,
runOrigin: 'system' as const,
};
if (oDoc.info?.logRemoveFrequency) {
await this.scheduleService.cancelIntervalTask(cron);
}
if (info.logRemoveFrequency && info.logRemoveFrequency > 0) {
this.scheduleService.createIntervalTask(cron, {
days: info.logRemoveFrequency,
});
this.scheduleService.createIntervalTask(
cron,
{
days: info.logRemoveFrequency,
},
true,
);
}
return { code: 200, data: info };
}
@@ -139,7 +158,10 @@ export default class SystemService {
}
let command = `cd && ${cmd}`;
const docs = await DependenceModel.findAll({
where: { type: DependenceTypes.nodejs, status: DependenceStatus.installed },
where: {
type: DependenceTypes.nodejs,
status: DependenceStatus.installed,
},
});
if (docs.length > 0) {
command += ` && pnpm i -g`;
@@ -166,6 +188,8 @@ export default class SystemService {
},
{
command,
id: 'update-node-mirror',
runOrigin: 'system',
},
);
}
@@ -196,6 +220,9 @@ export default class SystemService {
});
let defaultDomain = 'https://dl-cdn.alpinelinux.org';
let targetDomain = 'https://dl-cdn.alpinelinux.org';
if (os.platform() !== 'linux') {
return;
}
const content = await fs.promises.readFile('/etc/apk/repositories', {
encoding: 'utf-8',
});
@@ -237,6 +264,8 @@ export default class SystemService {
},
{
command,
id: 'update-linux-mirror',
runOrigin: 'system',
},
);
}
@@ -247,14 +276,18 @@ export default class SystemService {
let lastVersionContent;
try {
const result = await got.get(
const { body } = await request(
`${config.lastVersionFile}?t=${Date.now()}`,
{
timeout: 30000,
dispatcher: new Agent({
keepAliveTimeout: 30000,
keepAliveMaxTimeout: 30000,
}),
},
);
lastVersionContent = await parseContentVersion(result.body);
} catch (error) {}
const text = await body.text();
lastVersionContent = parseContentVersion(text);
} catch (error) { }
if (!lastVersionContent) {
lastVersionContent = currentVersionContent;
@@ -326,36 +359,41 @@ export default class SystemService {
return { code: 200 };
}
public async reloadSystem(target: 'system' | 'data') {
public async reloadSystem(target?: 'system' | 'data') {
const cmd = `real_time=true ql reload ${target || ''}`;
const cp = spawn(cmd, { shell: '/bin/bash' });
cp.stdout.on('data', (data) => {
this.sockService.sendMessage({
type: 'reloadSystem',
message: data.toString(),
});
const cp = spawn(cmd, {
shell: '/bin/bash',
detached: true,
stdio: 'ignore',
});
cp.stderr.on('data', (data) => {
this.sockService.sendMessage({
type: 'reloadSystem',
message: data.toString(),
});
cp.unref();
setTimeout(() => {
process.exit(0);
});
cp.on('error', (err) => {
this.sockService.sendMessage({
type: 'reloadSystem',
message: JSON.stringify(err),
});
});
return { code: 200 };
}
public async notify({ title, content }: { title: string; content: string }) {
const isSuccess = await this.notificationService.notify(title, content);
public async notify({
title,
content,
notificationInfo,
}: {
title: string;
content: string;
notificationInfo?: NotificationInfo;
}) {
const typeString =
typeof notificationInfo?.type === 'number'
? NotificationModeStringMap[notificationInfo.type]
: undefined;
if (notificationInfo && typeString) {
notificationInfo.type = typeString;
}
const isSuccess = await this.notificationService.notify(
title,
content,
notificationInfo,
);
if (isSuccess) {
return { code: 200, message: '通知发送成功' };
} else {
@@ -363,20 +401,16 @@ export default class SystemService {
}
}
public async run(
{ command, logPath }: { command: string; logPath: string },
callback: TaskCallbacks,
) {
public async run({ command, logPath }: { command: string; logPath?: string }, callback: TaskCallbacks) {
if (!command.startsWith(TASK_COMMAND)) {
command = `${TASK_COMMAND} ${command}`;
}
this.scheduleService.runTask(
`real_log_path=${logPath} real_time=true ${command}`,
callback,
{
command,
},
);
const logPathPrefix = logPath ? `real_log_path=${logPath}` : ''
this.scheduleService.runTask(`${logPathPrefix} real_time=true ${command}`, callback, {
command,
id: command.replace(/ /g, '-'),
runOrigin: 'system',
});
}
public async stop({ command, pid }: { command: string; pid: number }) {
@@ -401,11 +435,16 @@ export default class SystemService {
}
}
public async exportData(res: Response) {
public async exportData(res: Response, type?: string[]) {
try {
await tar.create(
{ gzip: true, file: config.dataTgzFile, cwd: config.rootPath },
['data'],
let dataDirs = ['db', 'upload'];
if (type && type.length) {
dataDirs = dataDirs.concat(type.filter((x) => x !== 'base'));
}
const dataPaths = dataDirs.map((dir) => `data/${dir}`);
await promiseExec(
`cd ${config.dataPath} && cd ../ && tar -zcvf ${config.dataTgzFile
} ${dataPaths.join(' ')}`,
);
res.download(config.dataTgzFile);
} catch (error: any) {
@@ -416,16 +455,34 @@ export default class SystemService {
public async importData() {
try {
await promiseExec(`rm -rf ${path.join(config.tmpPath, 'data')}`);
await tar.x({ file: config.dataTgzFile, cwd: config.tmpPath });
return { code: 200 };
const res = await promiseExec(
`cd ${config.tmpPath} && tar -zxvf ${config.dataTgzFile}`,
);
return { code: 200, data: res };
} catch (error: any) {
return { code: 400, message: error.message };
}
}
public async getSystemLog(res: Response) {
public async getSystemLog(
res: Response,
query: {
startTime?: string;
endTime?: string;
},
) {
const startTime = dayjs(query.startTime || undefined)
.startOf('d')
.valueOf();
const endTime = dayjs(query.endTime || undefined)
.endOf('d')
.valueOf();
const result = await readDirs(config.systemLogPath, config.systemLogPath);
const logs = result.reverse().filter((x) => x.title.endsWith('.log'));
const logs = result
.reverse()
.filter((x) => x.title.endsWith('.log'))
.filter((x) => x.createTime >= startTime && x.createTime <= endTime);
res.set({
'Content-Length': sum(logs.map((x) => x.size)),
});
@@ -447,4 +504,61 @@ export default class SystemService {
}
})(res, logs);
}
public async deleteSystemLog() {
const result = await readDirs(config.systemLogPath, config.systemLogPath);
const logs = result.reverse().filter((x) => x.title.endsWith('.log'));
for (const log of logs) {
await rmPath(path.join(config.systemLogPath, log.title));
}
}
public async updateTimezone(info: SystemModelInfo) {
if (!info.timezone) {
info.timezone = 'Asia/Shanghai';
}
const oDoc = await this.getSystemConfig();
await this.updateAuthDb({
...oDoc,
info: { ...oDoc.info, ...info },
});
const success = await setSystemTimezone(info.timezone);
if (success) {
return { code: 200, data: info };
} else {
return { code: 400, message: '设置时区失败' };
}
}
public async updateGlobalSshKey(info: SystemModelInfo) {
const oDoc = await this.getSystemConfig();
const result = await this.updateAuthDb({
...oDoc,
info: { ...oDoc.info, ...info },
});
// Apply the global SSH key
const SshKeyService = require('./sshKey').default;
const Container = require('typedi').Container;
const sshKeyService = Container.get(SshKeyService);
if (info.globalSshKey) {
await sshKeyService.addGlobalSSHKey(info.globalSshKey, 'global');
} else {
await sshKeyService.removeGlobalSSHKey('global');
}
return { code: 200, data: result };
}
public async cleanDependence(type: 'node' | 'python3') {
if (!type || !['node', 'python3'].includes(type)) {
return { code: 400, message: '参数错误' };
}
try {
const finalPath = path.join(config.dependenceCachePath, type);
await fs.promises.rm(finalPath, { recursive: true });
} catch (error) { }
return { code: 200 };
}
}
+321 -200
View File
@@ -1,22 +1,17 @@
import { Service, Inject } from 'typedi';
import winston from 'winston';
import {
createRandomString,
fileExist,
getNetIp,
getPlatform,
safeJSONParse,
} from '../config/util';
import { createRandomString } from '../config/util';
import config from '../config';
import * as fs from 'fs/promises';
import jwt from 'jsonwebtoken';
import { authenticator } from '@otplib/preset-default';
import {
AuthDataType,
AuthInfo,
SystemInfo,
SystemModel,
SystemModelInfo,
LoginStatus,
AuthInfo,
TokenInfo,
} from '../data/system';
import { NotificationInfo } from '../data/notify';
import NotificationService from './notify';
@@ -27,6 +22,9 @@ import dayjs from 'dayjs';
import IP2Region from 'ip2region';
import requestIp from 'request-ip';
import uniq from 'lodash/uniq';
import pickBy from 'lodash/pickBy';
import isNil from 'lodash/isNil';
import { shareStore } from '../shared/store';
@Service()
export default class UserService {
@@ -47,161 +45,184 @@ export default class UserService {
req: Request,
needTwoFactor = true,
): Promise<any> {
const _exist = await fileExist(config.authConfigFile);
if (!_exist) {
return this.initAuthInfo();
}
let { username, password } = payloads;
const content = await this.getAuthInfo();
const timestamp = Date.now();
if (content) {
let {
username: cUsername,
password: cPassword,
retries = 0,
lastlogon,
lastip,
lastaddr,
twoFactorActivated,
twoFactorActived,
tokens = {},
platform,
} = content;
// patch old field
twoFactorActivated = twoFactorActivated || twoFactorActived;
let {
username: cUsername,
password: cPassword,
retries = 0,
lastlogon,
lastip,
lastaddr,
twoFactorActivated,
tokens = {},
platform,
} = content;
const retriesTime = Math.pow(3, retries) * 1000;
if (retries > 2 && timestamp - lastlogon < retriesTime) {
const waitTime = Math.ceil(
(retriesTime - (timestamp - lastlogon)) / 1000,
);
return {
code: 410,
message: `失败次数过多,请${waitTime}秒后重试`,
data: waitTime,
};
}
if (
(cUsername === 'admin' && cPassword === 'admin') ||
!cUsername ||
!cPassword
) {
return this.initAuthInfo();
}
if (
username === cUsername &&
password === cPassword &&
twoFactorActivated &&
needTwoFactor
) {
await this.updateAuthInfo(content, {
isTwoFactorChecking: true,
});
return {
code: 420,
message: '',
};
}
const retriesTime = Math.pow(3, retries) * 1000;
if (retries > 2 && timestamp - lastlogon < retriesTime) {
const waitTime = Math.ceil(
(retriesTime - (timestamp - lastlogon)) / 1000,
);
const ip = requestIp.getClientIp(req) || '';
const query = new IP2Region();
const ipAddress = query.search(ip);
let address = '';
if (ipAddress) {
const { country, province, city, isp } = ipAddress;
address = uniq([country, province, city, isp]).filter(Boolean).join(' ');
}
if (username === cUsername && password === cPassword) {
const data = createRandomString(50, 100);
const expiration = twoFactorActivated ? '60d' : '20d';
let token = jwt.sign({ data }, config.jwt.secret, {
expiresIn: config.jwt.expiresIn || expiration,
algorithm: 'HS384',
});
const tokenInfo: TokenInfo = {
value: token,
timestamp,
ip,
address,
platform: req.platform,
};
const updatedTokens = this.addTokenToList(
tokens,
req.platform,
tokenInfo,
);
await this.updateAuthInfo(content, {
token,
tokens: updatedTokens,
lastlogon: timestamp,
retries: 0,
lastip: ip,
lastaddr: address,
platform: req.platform,
isTwoFactorChecking: false,
});
this.notificationService.notify(
'登录通知',
`你于${dayjs(timestamp).format('YYYY-MM-DD HH:mm:ss')}${address} ${
req.platform
} ip地址 ${ip}`,
);
await this.insertDb({
type: AuthDataType.loginLog,
info: {
timestamp,
address,
ip,
platform: req.platform,
status: LoginStatus.success,
},
});
this.getLoginLog();
return {
code: 200,
data: {
token,
lastip,
lastaddr,
lastlogon,
retries,
platform,
},
};
} else {
await this.updateAuthInfo(content, {
retries: retries + 1,
lastlogon: timestamp,
lastip: ip,
lastaddr: address,
platform: req.platform,
});
this.notificationService.notify(
'登录通知',
`你于${dayjs(timestamp).format('YYYY-MM-DD HH:mm:ss')}${address} ${
req.platform
} ip地址 ${ip}`,
);
await this.insertDb({
type: AuthDataType.loginLog,
info: {
timestamp,
address,
ip,
platform: req.platform,
status: LoginStatus.fail,
},
});
this.getLoginLog();
if (retries > 2) {
const waitTime = Math.round(Math.pow(3, retries + 1));
return {
code: 410,
message: `失败次数过多,请${waitTime}秒后重试`,
data: waitTime,
};
}
if (
username === cUsername &&
password === cPassword &&
twoFactorActivated &&
needTwoFactor
) {
this.updateAuthInfo(content, {
isTwoFactorChecking: true,
});
return {
code: 420,
message: '',
};
}
const ip = requestIp.getClientIp(req) || '';
const query = new IP2Region();
const ipAddress = query.search(ip);
let address = '';
if (ipAddress) {
const { country, province, city, isp } = ipAddress;
address = uniq([country, province, city, isp])
.filter(Boolean)
.join(' ');
}
if (username === cUsername && password === cPassword) {
const data = createRandomString(50, 100);
const expiration = twoFactorActivated ? 60 : 20;
let token = jwt.sign({ data }, config.secret as any, {
expiresIn: 60 * 60 * 24 * expiration,
algorithm: 'HS384',
});
this.updateAuthInfo(content, {
token,
tokens: {
...tokens,
[req.platform]: token,
},
lastlogon: timestamp,
retries: 0,
lastip: ip,
lastaddr: address,
platform: req.platform,
isTwoFactorChecking: false,
});
this.notificationService.notify(
'登录通知',
`你于${dayjs(timestamp).format('YYYY-MM-DD HH:mm:ss')}${address} ${
req.platform
} ip地址 ${ip}`,
);
await this.insertDb({
type: AuthDataType.loginLog,
info: {
timestamp,
address,
ip,
platform: req.platform,
status: LoginStatus.success,
},
});
return {
code: 200,
data: { token, lastip, lastaddr, lastlogon, retries, platform },
};
} else {
this.updateAuthInfo(content, {
retries: retries + 1,
lastlogon: timestamp,
lastip: ip,
lastaddr: address,
platform: req.platform,
});
this.notificationService.notify(
'登录通知',
`你于${dayjs(timestamp).format('YYYY-MM-DD HH:mm:ss')}${address} ${
req.platform
} ip地址 ${ip}`,
);
await this.insertDb({
type: AuthDataType.loginLog,
info: {
timestamp,
address,
ip,
platform: req.platform,
status: LoginStatus.fail,
},
});
if (retries > 2) {
const waitTime = Math.round(Math.pow(3, retries + 1));
return {
code: 410,
message: `失败次数过多,请${waitTime}秒后重试`,
data: waitTime,
};
} else {
return { code: 400, message: config.authError };
}
return { code: 400, message: config.authError };
}
} else {
return this.initAuthInfo();
}
}
public async logout(platform: string): Promise<any> {
public async logout(platform: string, tokenValue: string): Promise<any> {
if (!platform || !tokenValue) {
this.logger.warn('Invalid logout parameters - empty platform or token');
return;
}
const authInfo = await this.getAuthInfo();
this.updateAuthInfo(authInfo, {
token: '',
tokens: { ...authInfo.tokens, [platform]: '' },
// Verify the token exists before attempting to remove it
const tokenExists = this.findTokenInList(
authInfo.tokens,
platform,
tokenValue,
);
if (!tokenExists && authInfo.token !== tokenValue) {
// Token not found, but don't throw error - user may have already logged out
this.logger.info(
`Logout attempted for non-existent token on platform: ${platform}`,
);
return;
}
const updatedTokens = this.removeTokenFromList(
authInfo.tokens,
platform,
tokenValue,
);
await this.updateAuthInfo(authInfo, {
token: authInfo.token === tokenValue ? '' : authInfo.token,
tokens: updatedTokens,
});
}
@@ -214,8 +235,9 @@ export default class UserService {
(a, b) => b.info!.timestamp! - a.info!.timestamp!,
);
if (result.length > 100) {
const ids = result.slice(100).map((x) => x.id!);
await SystemModel.destroy({
where: { id: result[result.length - 1].id },
where: { id: ids },
});
}
return result.map((x) => x.info);
@@ -223,25 +245,11 @@ export default class UserService {
return [];
}
private async insertDb(payload: AuthInfo): Promise<AuthInfo> {
private async insertDb(payload: SystemInfo): Promise<SystemInfo> {
const doc = await SystemModel.create({ ...payload }, { returning: true });
return doc;
}
private async initAuthInfo() {
await fs.writeFile(
config.authConfigFile,
JSON.stringify({
username: 'admin',
password: 'admin',
}),
);
return {
code: 100,
message: '未找到认证文件,重新初始化',
};
}
public async updateUsernameAndPassword({
username,
password,
@@ -253,35 +261,21 @@ export default class UserService {
return { code: 400, message: '密码不能设置为admin' };
}
const authInfo = await this.getAuthInfo();
this.updateAuthInfo(authInfo, { username, password });
await this.updateAuthInfo(authInfo, { username, password });
return { code: 200, message: '更新成功' };
}
public async updateAvatar(avatar: string) {
const authInfo = await this.getAuthInfo();
this.updateAuthInfo(authInfo, { avatar });
await this.updateAuthInfo(authInfo, { avatar });
return { code: 200, data: avatar, message: '更新成功' };
}
public async getUserInfo(): Promise<any> {
const authFileExist = await fileExist(config.authConfigFile);
if (!authFileExist) {
await fs.writeFile(
config.authConfigFile,
JSON.stringify({
username: 'admin',
password: 'admin',
}),
);
}
return await this.getAuthInfo();
}
public async initTwoFactor() {
const secret = authenticator.generateSecret();
const authInfo = await this.getAuthInfo();
const otpauth = authenticator.keyuri(authInfo.username, 'qinglong', secret);
this.updateAuthInfo(authInfo, { twoFactorSecret: secret });
await this.updateAuthInfo(authInfo, { twoFactorSecret: secret });
return { secret, url: otpauth };
}
@@ -292,7 +286,7 @@ export default class UserService {
secret: authInfo.twoFactorSecret,
});
if (isValid) {
this.updateAuthInfo(authInfo, { twoFactorActivated: true });
await this.updateAuthInfo(authInfo, { twoFactorActivated: true });
}
return isValid;
}
@@ -317,8 +311,17 @@ export default class UserService {
if (isValid) {
return this.login({ username, password }, req, false);
} else {
const { ip, address } = await getNetIp(req);
this.updateAuthInfo(authInfo, {
const ip = requestIp.getClientIp(req) || '';
const query = new IP2Region();
const ipAddress = query.search(ip);
let address = '';
if (ipAddress) {
const { country, province, city, isp } = ipAddress;
address = uniq([country, province, city, isp])
.filter(Boolean)
.join(' ');
}
await this.updateAuthInfo(authInfo, {
lastip: ip,
lastaddr: address,
platform: req.platform,
@@ -329,33 +332,38 @@ export default class UserService {
public async deactiveTwoFactor() {
const authInfo = await this.getAuthInfo();
this.updateAuthInfo(authInfo, {
await this.updateAuthInfo(authInfo, {
twoFactorActivated: false,
twoFactorActived: false,
twoFactorSecret: '',
});
return true;
}
private async getAuthInfo() {
const content = await fs.readFile(config.authConfigFile, 'utf8');
return safeJSONParse(content);
public async getAuthInfo() {
const authInfo = await shareStore.getAuthInfo();
if (authInfo) {
return authInfo;
}
const doc = await this.getDb({ type: AuthDataType.authConfig });
return (doc.info || {}) as AuthInfo;
}
private async updateAuthInfo(authInfo: any, info: any) {
await fs.writeFile(
config.authConfigFile,
JSON.stringify({ ...authInfo, ...info }),
);
private async updateAuthInfo(authInfo: AuthInfo, info: Partial<AuthInfo>) {
const result = { ...authInfo, ...info };
await shareStore.updateAuthInfo(result);
await this.updateAuthDb({
type: AuthDataType.authConfig,
info: result,
});
}
public async getNotificationMode(): Promise<NotificationInfo> {
const doc = await this.getDb({ type: AuthDataType.notification });
return (doc && doc.info) || {};
return (doc.info || {}) as NotificationInfo;
}
private async updateAuthDb(payload: AuthInfo): Promise<any> {
let doc = await SystemModel.findOne({ type: payload.type });
private async updateAuthDb(payload: SystemInfo): Promise<any> {
let doc = await SystemModel.findOne({ where: { type: payload.type } });
if (doc) {
const updateResult = await SystemModel.update(payload, {
where: { id: doc.id },
@@ -368,9 +376,12 @@ export default class UserService {
return doc;
}
public async getDb(query: any): Promise<any> {
const doc: any = await SystemModel.findOne({ where: { ...query } });
return doc && (doc.get({ plain: true }) as any);
public async getDb(query: any): Promise<SystemInfo> {
const doc = await SystemModel.findOne({ where: { ...query } });
if (!doc) {
throw new Error(`${JSON.stringify(query)} not found`);
}
return doc.get({ plain: true });
}
public async updateNotificationMode(notificationInfo: NotificationInfo) {
@@ -390,4 +401,114 @@ export default class UserService {
return { code: 400, message: '通知发送失败,请检查参数' };
}
}
private normalizeTokens(
tokens: Record<string, string | TokenInfo[]>,
): Record<string, TokenInfo[]> {
const normalized: Record<string, TokenInfo[]> = {};
for (const [platform, value] of Object.entries(tokens)) {
if (typeof value === 'string') {
// Legacy format: convert string token to TokenInfo array
if (value) {
normalized[platform] = [
{
value,
timestamp: Date.now(),
ip: '',
address: '',
platform,
},
];
} else {
normalized[platform] = [];
}
} else {
// Already in new format
normalized[platform] = value || [];
}
}
return normalized;
}
private addTokenToList(
tokens: Record<string, string | TokenInfo[]>,
platform: string,
tokenInfo: TokenInfo,
maxTokensPerPlatform: number = config.maxTokensPerPlatform,
): Record<string, TokenInfo[]> {
// Validate maxTokensPerPlatform parameter
if (!Number.isInteger(maxTokensPerPlatform) || maxTokensPerPlatform < 1) {
this.logger.warn(
`Invalid maxTokensPerPlatform value: ${maxTokensPerPlatform}, using default`,
);
maxTokensPerPlatform = config.maxTokensPerPlatform;
}
const normalized = this.normalizeTokens(tokens);
if (!normalized[platform]) {
normalized[platform] = [];
}
// Add new token
normalized[platform].unshift(tokenInfo);
// Limit the number of active tokens per platform
if (normalized[platform].length > maxTokensPerPlatform) {
normalized[platform] = normalized[platform].slice(
0,
maxTokensPerPlatform,
);
}
return normalized;
}
private removeTokenFromList(
tokens: Record<string, string | TokenInfo[]>,
platform: string,
tokenValue: string,
): Record<string, TokenInfo[]> {
const normalized = this.normalizeTokens(tokens);
if (normalized[platform]) {
normalized[platform] = normalized[platform].filter(
(t) => t.value !== tokenValue,
);
}
return normalized;
}
private findTokenInList(
tokens: Record<string, string | TokenInfo[]>,
platform: string,
tokenValue: string,
): TokenInfo | undefined {
const normalized = this.normalizeTokens(tokens);
if (normalized[platform]) {
return normalized[platform].find((t) => t.value === tokenValue);
}
return undefined;
}
public async resetAuthInfo(info: Partial<AuthInfo>) {
const { retries, twoFactorActivated, password, username } = info;
const authInfo = await this.getAuthInfo();
const payload = pickBy(
{
retries,
twoFactorActivated,
password,
username,
},
(x) => !isNil(x),
);
await this.updateAuthInfo(authInfo, payload);
}
}
+46
View File
@@ -0,0 +1,46 @@
import { AuthInfo, TokenInfo } from '../data/system';
/**
* Validates if a token exists in the authentication info.
* Supports both legacy string tokens and new TokenInfo array format.
*
* @param authInfo - The authentication information
* @param headerToken - The token to validate
* @param platform - The platform (desktop, mobile)
* @returns true if the token is valid, false otherwise
*/
export function isValidToken(
authInfo: AuthInfo | null | undefined,
headerToken: string,
platform: string,
): boolean {
if (!authInfo || !headerToken) {
return false;
}
const { token = '', tokens = {} } = authInfo;
// Check legacy token field
if (headerToken === token) {
return true;
}
// Check platform-specific tokens (support both legacy string and new TokenInfo[] format)
const platformTokens = tokens[platform];
// Handle null/undefined platformTokens
if (platformTokens === null || platformTokens === undefined) {
return false;
}
if (typeof platformTokens === 'string') {
// Legacy format: single string token
return headerToken === platformTokens;
} else if (Array.isArray(platformTokens)) {
// New format: array of TokenInfo objects
return platformTokens.some((t: TokenInfo) => t && t.value === headerToken);
}
// Unexpected type - log warning and reject
return false;
}
+33
View File
@@ -0,0 +1,33 @@
import { Dependence } from '../data/dependence';
import { ICron } from '../protos/cron';
export type Override<
T,
K extends Partial<{ [P in keyof T]: any }> | string,
> = K extends string
? Omit<T, K> & { [P in keyof T]: T[P] | unknown }
: Omit<T, keyof K> & K;
export type TCron = Override<Partial<ICron>, { id: string }>;
export interface IDependencyFn<T> {
(): Promise<T>;
dependency?: Dependence;
}
export interface ICronFn<T> {
(): Promise<T>;
cron?: TCron;
}
export interface ISchedule {
schedule?: string;
name?: string;
command?: string;
id: string;
}
export interface IScheduleFn<T> {
(): Promise<T>;
schedule?: ISchedule;
}
+110
View File
@@ -0,0 +1,110 @@
import { createWriteStream, WriteStream } from 'fs';
import { EventEmitter } from 'events';
/**
* Manages write streams for log files to improve performance by avoiding repeated file opens
*/
export class LogStreamManager extends EventEmitter {
private streams: Map<string, WriteStream> = new Map();
private pendingWrites: Map<string, Promise<void>> = new Map();
/**
* Write data to a log file using a managed stream
* @param filePath - Absolute path to the log file
* @param data - Data to write to the log file
*/
async write(filePath: string, data: string): Promise<void> {
// Wait for any pending writes to this file to complete
const pending = this.pendingWrites.get(filePath);
if (pending) {
await pending;
}
// Create a new promise for this write operation
const writePromise = new Promise<void>((resolve, reject) => {
let stream = this.streams.get(filePath);
if (!stream) {
// Create a new write stream if one doesn't exist
stream = createWriteStream(filePath, { flags: 'a' });
this.streams.set(filePath, stream);
// Handle stream errors
stream.on('error', (error) => {
this.emit('error', { filePath, error });
// Remove the stream from the map on error
this.streams.delete(filePath);
reject(error);
});
}
// Write the data
const canContinue = stream.write(data, 'utf8', (error) => {
if (error) {
reject(error);
} else {
resolve();
}
});
// Handle backpressure
if (!canContinue) {
stream.once('drain', () => {
// Stream is ready for more data
});
}
});
this.pendingWrites.set(filePath, writePromise);
try {
await writePromise;
} finally {
this.pendingWrites.delete(filePath);
}
}
/**
* Close the stream for a specific file path
* @param filePath - Absolute path to the log file
*/
async closeStream(filePath: string): Promise<void> {
// Wait for any pending writes to complete
const pending = this.pendingWrites.get(filePath);
if (pending) {
await pending.catch(() => {
// Ignore errors on pending writes during close
});
}
const stream = this.streams.get(filePath);
if (stream) {
return new Promise<void>((resolve) => {
stream.end(() => {
this.streams.delete(filePath);
resolve();
});
});
}
}
/**
* Close all open streams
*/
async closeAll(): Promise<void> {
const closePromises = Array.from(this.streams.keys()).map((filePath) =>
this.closeStream(filePath),
);
await Promise.all(closePromises);
}
/**
* Get the number of open streams
*/
getOpenStreamCount(): number {
return this.streams.size;
}
}
// Export a singleton instance for shared use
export const logStreamManager = new LogStreamManager();
+118 -5
View File
@@ -3,18 +3,44 @@ import os from 'os';
import { AuthDataType, SystemModel } from '../data/system';
import Logger from '../loaders/logger';
import { Dependence } from '../data/dependence';
import NotificationService from '../services/notify';
import {
ICronFn,
IDependencyFn,
ISchedule,
IScheduleFn,
TCron,
} from './interface';
import config from '../config';
import { credentials } from '@grpc/grpc-js';
import { ApiClient } from '../protos/api';
interface IDependencyFn<T> {
(): Promise<T>;
dependency?: Dependence;
}
class TaskLimit {
private dependenyLimit = new PQueue({ concurrency: 1 });
private queuedDependencyIds = new Set<number>([]);
private queuedCrons = new Map<string, ICronFn<any>[]>();
private repeatCronNotifyMap = new Map<string, number>();
private updateLogLimit = new PQueue({ concurrency: 1 });
private cronLimit = new PQueue({
concurrency: Math.max(os.cpus().length, 4),
});
private manualCronoLimit = new PQueue({
concurrency: Math.max(os.cpus().length, 4),
});
private subscriptionLimit = new PQueue({
concurrency: Math.max(os.cpus().length, 4),
});
private scriptLimit = new PQueue({
concurrency: Math.max(os.cpus().length, 4),
});
private systemLimit = new PQueue({
concurrency: Math.max(os.cpus().length, 4),
});
private client = new ApiClient(
`0.0.0.0:${config.grpcPort}`,
credentials.createInsecure(),
{ 'grpc.enable_http_proxy': 0 },
);
get cronLimitActiveCount() {
return this.cronLimit.pending;
@@ -28,6 +54,8 @@ class TaskLimit {
return [...this.queuedDependencyIds.values()][0];
}
private notificationService: NotificationService = new NotificationService();
constructor() {
this.setCustomLimit();
this.handleEvents();
@@ -68,9 +96,20 @@ class TaskLimit {
}
}
public removeQueuedCron(id: string) {
if (this.queuedCrons.has(id)) {
const runs = this.queuedCrons.get(id);
if (runs && runs.length > 0) {
runs.pop();
this.queuedCrons.set(id, runs);
}
}
}
public async setCustomLimit(limit?: number) {
if (limit) {
this.cronLimit.concurrency = limit;
this.manualCronoLimit.concurrency = limit;
return;
}
await SystemModel.sync();
@@ -79,14 +118,88 @@ class TaskLimit {
});
if (doc?.info?.cronConcurrency) {
this.cronLimit.concurrency = doc.info.cronConcurrency;
this.manualCronoLimit.concurrency = doc.info.cronConcurrency;
}
}
public async runWithCronLimit<T>(
cron: TCron,
fn: ICronFn<T>,
options?: Partial<QueueAddOptions>,
): Promise<T | void> {
fn.cron = cron;
let runs = this.queuedCrons.get(cron.id);
const result = runs?.length ? [...runs, fn] : [fn];
const repeatTimes = this.repeatCronNotifyMap.get(cron.id) || 0;
if (result?.length > 5) {
if (repeatTimes < 3) {
this.repeatCronNotifyMap.set(cron.id, repeatTimes + 1);
this.client.systemNotify(
{
title: '任务重复运行',
content: `任务:${cron.name},命令:${cron.command},定时:${cron.schedule},处于运行中的超过 5 个,请检查定时设置`,
},
(err, res) => {
if (err) {
Logger.error(
`[schedule][任务重复运行] 通知失败 ${JSON.stringify(err)}`,
);
}
},
);
}
Logger.warn(`[schedule][任务重复运行] 参数 ${JSON.stringify(cron)}`);
return;
}
this.queuedCrons.set(cron.id, result);
return this.cronLimit.add(fn, options);
}
public async manualRunWithCronLimit<T>(
fn: () => Promise<T>,
options?: Partial<QueueAddOptions>,
): Promise<T | void> {
return this.cronLimit.add(fn, options);
return this.manualCronoLimit.add(fn, options);
}
public async runWithSubscriptionLimit<T>(
schedule: TCron,
fn: IScheduleFn<T>,
options?: Partial<QueueAddOptions>,
): Promise<T | void> {
fn.schedule = schedule;
return this.subscriptionLimit.add(fn, options);
}
public async runWithSystemLimit<T>(
schedule: TCron,
fn: IScheduleFn<T>,
options?: Partial<QueueAddOptions>,
): Promise<T | void> {
fn.schedule = schedule;
return this.systemLimit.add(fn, options);
}
public async runWithScriptLimit<T>(
schedule: ISchedule,
fn: IScheduleFn<T>,
options?: Partial<QueueAddOptions>,
): Promise<T | void> {
fn.schedule = schedule;
return this.scriptLimit.add(fn, options);
}
public async waitDependencyQueueDone(): Promise<void> {
if (this.dependenyLimit.size === 0 && this.dependenyLimit.pending === 0) {
return;
}
return new Promise((resolve) => {
const onIdle = () => {
this.dependenyLimit.removeListener('idle', onIdle);
resolve();
};
this.dependenyLimit.on('idle', onIdle);
});
}
public runDependeny<T>(
+54 -4
View File
@@ -1,11 +1,52 @@
import { spawn } from 'cross-spawn';
import taskLimit from './pLimit';
import Logger from '../loaders/logger';
import { ICron } from '../protos/cron';
import { CrontabModel, CrontabStatus } from '../data/cron';
import { killTask } from '../config/util';
export function runCron(cmd: string, options?: { schedule: string; extraSchedules: Array<{ schedule: string }>; name: string }): Promise<number | void> {
return taskLimit.runWithCronLimit(() => {
export function runCron(cmd: string, cron: ICron): Promise<number | void> {
return taskLimit.runWithCronLimit(cron, () => {
return new Promise(async (resolve: any) => {
Logger.info(`[schedule][开始执行任务] 参数 ${JSON.stringify({ ...options, command: cmd })}`);
// Check if the cron is already running and stop it (only if multiple instances are not allowed)
try {
const existingCron = await CrontabModel.findOne({
where: { id: Number(cron.id) },
});
// Default to single instance mode (0) for backward compatibility
const allowSingleInstances =
existingCron?.allow_multiple_instances === 0;
if (
allowSingleInstances &&
existingCron &&
existingCron.pid &&
(existingCron.status === CrontabStatus.running ||
existingCron.status === CrontabStatus.queued)
) {
Logger.info(
`[schedule][停止已运行任务] 任务ID: ${cron.id}, PID: ${existingCron.pid}`,
);
await killTask(existingCron.pid);
// Update the status to idle after killing
await CrontabModel.update(
{ status: CrontabStatus.idle, pid: undefined },
{ where: { id: Number(cron.id) } },
);
}
} catch (error) {
Logger.error(
`[schedule][检查已运行任务失败] 任务ID: ${cron.id}, 错误: ${error}`,
);
}
Logger.info(
`[schedule][开始执行任务] 参数 ${JSON.stringify({
...cron,
command: cmd,
})}`,
);
const cp = spawn(cmd, { shell: '/bin/bash' });
cp.stderr.on('data', (data) => {
@@ -24,7 +65,16 @@ export function runCron(cmd: string, options?: { schedule: string; extraSchedule
});
cp.on('exit', async (code) => {
resolve({ ...options, command: cmd, pid: cp.pid, code });
taskLimit.removeQueuedCron(cron.id);
Logger.info(
'[schedule][执行任务结束] 参数: %s, 退出码: %j',
JSON.stringify({
...cron,
command: cmd,
}),
code,
);
resolve({ ...cron, command: cmd, pid: cp.pid, code });
});
});
});
+34
View File
@@ -0,0 +1,34 @@
import { AuthInfo } from '../data/system';
import { App } from '../data/open';
import Keyv from 'keyv';
import KeyvSqlite from '@keyv/sqlite';
import config from '../config';
import path from 'path';
export enum EKeyv {
'apps' = 'apps',
'authInfo' = 'authInfo',
}
export interface IKeyvStore {
apps: App[];
authInfo: AuthInfo;
}
const keyvSqlite = new KeyvSqlite(path.join(config.dbPath, 'keyv.sqlite'));
export const keyvStore = new Keyv<IKeyvStore>({ store: keyvSqlite });
export const shareStore = {
getAuthInfo() {
return keyvStore.get<IKeyvStore['authInfo']>(EKeyv.authInfo);
},
updateAuthInfo(value: IKeyvStore['authInfo']) {
return keyvStore.set<IKeyvStore['authInfo']>(EKeyv.authInfo, value);
},
getApps() {
return keyvStore.get<IKeyvStore['apps']>(EKeyv.apps);
},
updateApps(apps: App[]) {
return keyvStore.set<IKeyvStore['apps']>(EKeyv.apps, apps);
},
};
+42
View File
@@ -0,0 +1,42 @@
import { lock } from 'proper-lockfile';
import os from 'os';
import path from 'path';
import { writeFile, open, chmod } from 'fs/promises';
import { fileExist } from '../config/util';
function getUniqueLockPath(filePath: string) {
const sanitizedPath = filePath
.replace(/[<>:"/\\|?*]/g, '_')
.replace(/^_/, '');
return path.join(os.tmpdir(), `${sanitizedPath}.ql_lock`);
}
export async function writeFileWithLock(
filePath: string,
content: string,
options: Parameters<typeof writeFile>[2] = {},
) {
if (typeof options === 'string') {
options = { encoding: options };
}
if (!(await fileExist(filePath))) {
const fileHandle = await open(filePath, 'w');
fileHandle.close();
}
const lockfilePath = getUniqueLockPath(filePath);
const release = await lock(filePath, {
retries: {
retries: 10,
factor: 2,
minTimeout: 100,
maxTimeout: 3000,
},
lockfilePath,
});
await writeFile(filePath, content, { encoding: 'utf8', ...options });
if (options?.mode) {
await chmod(filePath, options.mode);
}
await release();
}
+2 -11
View File
@@ -2,10 +2,10 @@ import 'reflect-metadata';
import OpenService from './services/open';
import { Container } from 'typedi';
import LoggerInstance from './loaders/logger';
import fs from 'fs';
import config from './config';
import path from 'path';
import os from 'os';
import { writeFileWithLock } from './shared/utils';
const tokenFile = path.join(config.configPath, 'token.json');
@@ -25,16 +25,7 @@ async function getToken() {
}
async function writeFile(data: any) {
return new Promise<void>((resolve, reject) => {
fs.writeFile(
tokenFile,
`${JSON.stringify(data)}${os.EOL}`,
{ encoding: 'utf8' },
() => {
resolve();
},
);
});
await writeFileWithLock(tokenFile, `${JSON.stringify(data)}${os.EOL}`);
}
getToken();
+8 -8
View File
@@ -2,10 +2,11 @@
"compilerOptions": {
"target": "es2017",
"lib": ["ESNext"],
"typeRoots": ["./node_modules/celebrate/lib", "./node_modules/@types"],
"paths": {
"@/*": ["./back/*"]
},
"typeRoots": [
"./types",
"../node_modules/celebrate/lib",
"../node_modules/@types"
],
"allowSyntheticDefaultImports": true,
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
@@ -16,12 +17,11 @@
"module": "commonjs",
"pretty": true,
"sourceMap": true,
"outDir": "./static/build",
"outDir": "../static/build",
"allowJs": true,
"noEmit": false,
"esModuleInterop": true
},
"include": ["./back/**/*"],
"exclude": ["node_modules"],
"files": ["./back/index.d.ts"]
"include": ["./**/*"],
"exclude": ["node_modules"]
}
+11
View File
@@ -0,0 +1,11 @@
/// <reference types="express" />
export {};
declare global {
namespace Express {
interface Request {
platform: 'desktop' | 'mobile';
}
}
}
-24
View File
@@ -1,24 +0,0 @@
import 'reflect-metadata'; // We need this in order to use @Decorators
import config from './config';
import express from 'express';
import Logger from './loaders/logger';
async function startServer() {
const app = express();
await require('./loaders/update').default({ app });
app
.listen(config.updatePort, () => {
Logger.debug(`✌️ 更新服务启动成功!`);
console.debug(`✌️ 更新服务启动成功!`);
process.send?.('ready');
})
.on('error', (err) => {
Logger.error(err);
console.error(err);
process.exit(1);
});
}
startServer();
+85
View File
@@ -0,0 +1,85 @@
import { Joi } from 'celebrate';
import CronExpressionParser from 'cron-parser';
import { ScheduleType } from '../interface/schedule';
import path from 'path';
import config from '../config';
const validateSchedule = (value: string, helpers: any) => {
if (
value.startsWith(ScheduleType.ONCE) ||
value.startsWith(ScheduleType.BOOT)
) {
return value;
}
try {
if (CronExpressionParser.parse(value).hasNext()) {
return value;
}
} catch (e) {
return helpers.error('any.invalid');
}
return helpers.error('any.invalid');
};
export const scheduleSchema = Joi.string()
.required()
.custom(validateSchedule)
.messages({
'any.invalid': '无效的定时规则',
'string.empty': '定时规则不能为空',
});
export const commonCronSchema = {
name: Joi.string().optional(),
command: Joi.string().required(),
schedule: scheduleSchema,
labels: Joi.array().optional(),
sub_id: Joi.number().optional().allow(null),
extra_schedules: Joi.array().optional().allow(null),
task_before: Joi.string().optional().allow('').allow(null),
task_after: Joi.string().optional().allow('').allow(null),
log_name: Joi.string()
.optional()
.allow('')
.allow(null)
.custom((value, helpers) => {
if (!value) return value;
// Check if it's an absolute path
if (value.startsWith('/')) {
// Allow /dev/null as special case
if (value === '/dev/null') {
return value;
}
// For other absolute paths, ensure they are within the safe log directory
const normalizedValue = path.normalize(value);
const normalizedLogPath = path.normalize(config.logPath);
if (!normalizedValue.startsWith(normalizedLogPath)) {
return helpers.error('string.unsafePath');
}
return value;
}
if (
!/^(?!.*(?:^|\/)\.{1,2}(?:\/|$))(?:\/)?(?:[\w.-]+\/)*[\w.-]+\/?$/.test(
value,
)
) {
return helpers.error('string.pattern.base');
}
if (value.length > 100) {
return helpers.error('string.max');
}
return value;
})
.messages({
'string.pattern.base': '日志名称只能包含字母、数字、下划线和连字符',
'string.max': '日志名称不能超过100个字符',
'string.unsafePath': '绝对路径必须在日志目录内或使用 /dev/null',
}),
allow_multiple_instances: Joi.number().optional().valid(0, 1).allow(null),
};
-13
View File
@@ -1,13 +0,0 @@
import { CommandModule } from 'yargs';
export const updateCommand: CommandModule = {
command: 'update',
describe: 'Update and restart qinglong',
builder: (yargs) => {
return yargs.option('repositority', {
type: 'string',
alias: 'r',
describe: `Specify the release warehouse address of the package`,
});
},
handler: async (argv) => {},
};
-13
View File
@@ -1,13 +0,0 @@
import * as yargs from 'yargs';
import { green, red } from 'chalk';
import { updateCommand } from './commands/update';
yargs
.usage('Usage: ql [command] <options>')
.command(updateCommand)
.fail((err) => {
console.error(`${red(err)}`);
})
.alias('h', 'help')
.showHelp()
.recommendCommands().argv;
+32 -17
View File
@@ -1,27 +1,32 @@
FROM python:3.10-alpine3.18 as builder
FROM python:3.10-alpine3.18 AS builder
COPY package.json .npmrc pnpm-lock.yaml /tmp/build/
RUN set -x \
&& apk update \
&& apk add nodejs npm git \
&& npm i -g pnpm@8.3.1 \
&& npm i -g pnpm@8.3.1 pm2 ts-node \
&& cd /tmp/build \
&& pnpm install --prod
FROM python:3.10-alpine3.18
FROM python:3.10-alpine
ARG QL_MAINTAINER="whyour"
LABEL maintainer="${QL_MAINTAINER}"
ARG QL_URL=https://github.com/${QL_MAINTAINER}/qinglong.git
ARG QL_BRANCH=develop
ARG PYTHON_SHORT_VERSION=3.10
ENV PNPM_HOME=/root/.local/share/pnpm \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/share/pnpm:/root/.local/share/pnpm/global/5/node_modules:$PNPM_HOME \
NODE_PATH=/usr/local/bin:/usr/local/pnpm-global/5/node_modules:/usr/local/lib/node_modules:/root/.local/share/pnpm/global/5/node_modules \
ENV QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH} \
LANG=C.UTF-8 \
SHELL=/bin/bash \
PS1="\u@\h:\w \$ " \
QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH}
PS1="\u@\h:\w \$ "
VOLUME /ql/data
EXPOSE 5700
COPY --from=builder /usr/local/lib/node_modules/. /usr/local/lib/node_modules/
COPY --from=builder /usr/local/bin/. /usr/local/bin/
RUN set -x \
&& apk update -f \
@@ -34,7 +39,6 @@ RUN set -x \
tzdata \
perl \
openssl \
nginx \
nodejs \
jq \
openssh \
@@ -46,14 +50,10 @@ RUN set -x \
&& apk update \
&& ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezone \
&& git config --global user.email "qinglong@@users.noreply.github.com" \
&& git config --global user.email "qinglong@users.noreply.github.com" \
&& git config --global user.name "qinglong" \
&& git config --global http.postBuffer 524288000 \
&& npm install -g pnpm@8.3.1 pm2 tsx \
&& rm -rf /root/.pnpm-store \
&& rm -rf /root/.local/share/pnpm/store \
&& rm -rf /root/.cache \
&& rm -rf /root/.npm \
&& ulimit -c 0
ARG SOURCE_COMMIT
@@ -65,13 +65,28 @@ RUN git clone --depth=1 -b ${QL_BRANCH} ${QL_URL} ${QL_DIR} \
&& git clone --depth=1 -b ${QL_BRANCH} https://github.com/${QL_MAINTAINER}/qinglong-static.git /static \
&& mkdir -p ${QL_DIR}/static \
&& cp -rf /static/* ${QL_DIR}/static \
&& rm -rf /static
&& rm -rf /static \
&& mkdir -p ${QL_DIR}/.tmp ${QL_DIR}/shell/preload \
&& chmod -R a+w ${QL_DIR}/.tmp ${QL_DIR}/shell/preload ${QL_DIR}/static
ENV PNPM_HOME=${QL_DIR}/data/dep_cache/node \
PYTHON_HOME=${QL_DIR}/data/dep_cache/python3 \
PYTHONUSERBASE=${QL_DIR}/data/dep_cache/python3 \
PM2_HOME=${QL_DIR}/data/.pm2 \
HOME=/root
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PNPM_HOME}:${PYTHON_HOME}/bin:${HOME}/bin \
NODE_PATH=/usr/local/bin:/usr/local/lib/node_modules:${PNPM_HOME}/global/5/node_modules \
PIP_CACHE_DIR=${PYTHON_HOME}/pip \
PYTHONPATH=${PYTHON_HOME}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}/site-packages
RUN pip3 install --prefix ${PYTHON_HOME} requests
COPY --from=builder /tmp/build/node_modules/. /ql/node_modules/
WORKDIR ${QL_DIR}
HEALTHCHECK --interval=5s --timeout=2s --retries=20 \
CMD curl -sf --noproxy '*' http://127.0.0.1:5400/api/health || exit 1
CMD curl -sf --noproxy '*' http://127.0.0.1:${QlPort:-5700}/api/health || exit 1
ENTRYPOINT ["./docker/docker-entrypoint.sh"]
+24 -14
View File
@@ -1,9 +1,9 @@
FROM python:3.11-alpine3.18 as builder
FROM python:3.11-alpine3.18 AS builder
COPY package.json .npmrc pnpm-lock.yaml /tmp/build/
RUN set -x \
&& apk update \
&& apk add nodejs npm git \
&& npm i -g pnpm@8.3.1 pm2 tsx \
&& npm i -g pnpm@8.3.1 pm2 ts-node \
&& cd /tmp/build \
&& pnpm install --prod
@@ -13,15 +13,13 @@ ARG QL_MAINTAINER="whyour"
LABEL maintainer="${QL_MAINTAINER}"
ARG QL_URL=https://github.com/${QL_MAINTAINER}/qinglong.git
ARG QL_BRANCH=develop
ARG PYTHON_SHORT_VERSION=3.11
ENV PNPM_HOME=/root/.local/share/pnpm \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/share/pnpm:/root/.local/share/pnpm/global/5/node_modules:$PNPM_HOME \
NODE_PATH=/usr/local/bin:/usr/local/pnpm-global/5/node_modules:/usr/local/lib/node_modules:/root/.local/share/pnpm/global/5/node_modules \
ENV QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH} \
LANG=C.UTF-8 \
SHELL=/bin/bash \
PS1="\u@\h:\w \$ " \
QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH}
PS1="\u@\h:\w \$ "
VOLUME /ql/data
@@ -41,7 +39,6 @@ RUN set -x \
tzdata \
perl \
openssl \
nginx \
nodejs \
jq \
openssh \
@@ -53,11 +50,9 @@ RUN set -x \
&& apk update \
&& ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezone \
&& git config --global user.email "qinglong@@users.noreply.github.com" \
&& git config --global user.email "qinglong@users.noreply.github.com" \
&& git config --global user.name "qinglong" \
&& git config --global http.postBuffer 524288000 \
&& rm -rf /root/.pnpm-store \
&& rm -rf /root/.local/share/pnpm/store \
&& rm -rf /root/.cache \
&& ulimit -c 0
@@ -70,13 +65,28 @@ RUN git clone --depth=1 -b ${QL_BRANCH} ${QL_URL} ${QL_DIR} \
&& git clone --depth=1 -b ${QL_BRANCH} https://github.com/${QL_MAINTAINER}/qinglong-static.git /static \
&& mkdir -p ${QL_DIR}/static \
&& cp -rf /static/* ${QL_DIR}/static \
&& rm -rf /static
&& rm -rf /static \
&& mkdir -p ${QL_DIR}/.tmp ${QL_DIR}/shell/preload \
&& chmod -R a+w ${QL_DIR}/.tmp ${QL_DIR}/shell/preload ${QL_DIR}/static
ENV PNPM_HOME=${QL_DIR}/data/dep_cache/node \
PYTHON_HOME=${QL_DIR}/data/dep_cache/python3 \
PYTHONUSERBASE=${QL_DIR}/data/dep_cache/python3 \
PM2_HOME=${QL_DIR}/data/.pm2 \
HOME=/root
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PNPM_HOME}:${PYTHON_HOME}/bin:${HOME}/bin \
NODE_PATH=/usr/local/bin:/usr/local/lib/node_modules:${PNPM_HOME}/global/5/node_modules \
PIP_CACHE_DIR=${PYTHON_HOME}/pip \
PYTHONPATH=${PYTHON_HOME}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}/site-packages
RUN pip3 install --prefix ${PYTHON_HOME} requests
COPY --from=builder /tmp/build/node_modules/. /ql/node_modules/
WORKDIR ${QL_DIR}
HEALTHCHECK --interval=5s --timeout=2s --retries=20 \
CMD curl -sf --noproxy '*' http://127.0.0.1:5400/api/health || exit 1
CMD curl -sf --noproxy '*' http://127.0.0.1:${QlPort:-5700}/api/health || exit 1
ENTRYPOINT ["./docker/docker-entrypoint.sh"]
+3 -8
View File
@@ -1,15 +1,10 @@
version: '2'
services:
web:
# alpine 基础镜像版本
image: whyour/qinglong:latest
# debian-slim 基础镜像版本
# image: whyour/qinglong:debian
image: whyour/qinglong:latest # 基于 Debian 的版本:whyour/qinglong:debian
volumes:
- ./data:/ql/data
ports:
- "0.0.0.0:5700:5700"
- "5700:5700"
environment:
# 部署路径非必须,以斜杠开头和结尾,比如 /test/
QlBaseUrl: '/'
QlBaseUrl: '/' # 部署路径非必须,以斜杠开头和结尾,比如 /test/
restart: unless-stopped
+39 -23
View File
@@ -1,44 +1,60 @@
#!/bin/bash
dir_shell=/ql/shell
. $dir_shell/env.sh
. $dir_shell/share.sh
link_shell
echo -e "======================1. 检测配置文件========================\n"
make_dir /etc/nginx/conf.d
make_dir /run/nginx
init_nginx
export_ql_envs() {
export BACK_PORT="${ql_port}"
export GRPC_PORT="${ql_grpc_port}"
}
log_with_style() {
local level="$1"
local message="$2"
local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
printf "\n[%s] [%7s] %s\n" "${timestamp}" "${level}" "${message}"
}
# Fix DNS resolution issues in Alpine Linux (requires root)
# Alpine uses musl libc which has known DNS resolver issues with certain domains
# Adding ndots:0 prevents unnecessary search domain appending
if [ "$(id -u)" = "0" ] && [ -f /etc/alpine-release ]; then
if ! grep -q "^options ndots:0" /etc/resolv.conf 2>/dev/null; then
echo "options ndots:0" >> /etc/resolv.conf
log_with_style "INFO" "🔧 0. 已配置 DNS 解析优化 (ndots:0)"
fi
fi
log_with_style "INFO" "🚀 1. 检测配置文件..."
load_ql_envs
export_ql_envs
. $dir_shell/env.sh
import_config "$@"
fix_config
pm2 l &>/dev/null
# Try to initialize PM2, but don't fail if it doesn't work
pm2 l &>/dev/null || log_with_style "WARN" "PM2 初始化可能失败,将在启动时尝试使用备用方案"
echo -e "======================2. 安装依赖========================\n"
patch_version
echo -e "======================3. 启动nginx========================\n"
nginx -s reload 2>/dev/null || nginx -c /etc/nginx/nginx.conf
echo -e "nginx启动成功...\n"
echo -e "======================4. 启动pm2服务========================\n"
log_with_style "INFO" "⚙️ 2. 启动 pm2 服务..."
reload_pm2
if [[ $AutoStartBot == true ]]; then
echo -e "======================5. 启动bot========================\n"
log_with_style "INFO" "🤖 3. 启动 bot..."
nohup ql bot >$dir_log/bot.log 2>&1 &
echo -e "bot后台启动中...\n"
fi
if [[ $EnableExtraShell == true ]]; then
echo -e "====================6. 执行自定义脚本========================\n"
log_with_style "INFO" "🛠️ 4. 执行自定义脚本..."
nohup ql extra >$dir_log/extra.log 2>&1 &
echo -e "自定义脚本后台执行中...\n"
fi
echo -e "############################################################\n"
echo -e "容器启动成功..."
echo -e "############################################################\n"
log_with_style "SUCCESS" "🎉 容器启动成功!"
crond -f >/dev/null
if [ "$(id -u)" = "0" ]; then
crond -f >/dev/null
else
# crond requires root in Alpine; keep container alive without it
exec tail -f /dev/null
fi
exec "$@"
-93
View File
@@ -1,93 +0,0 @@
upstream baseApi {
server 0.0.0.0:5600;
}
upstream publicApi {
server 0.0.0.0:5400;
}
upstream updateApi {
server 0.0.0.0:5300;
}
map $http_upgrade $connection_upgrade {
default keep-alive;
'websocket' upgrade;
}
server {
IPV4_CONFIG
IPV6_CONFIG
ssl_session_timeout 5m;
location QL_BASE_URLapi/update/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://updateApi/api/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
}
location QL_BASE_URLapi/public/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://publicApi/api/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
}
location QL_BASE_URLapi/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://baseApi/api/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location QL_BASE_URLopen/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://baseApi/open/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
}
gzip on;
gzip_static on;
gzip_types text/plain application/json application/javascript application/x-javascript text/css application/xml text/javascript;
gzip_proxied any;
gzip_vary on;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.0;
QL_ROOT_CONFIG
location QL_BASE_URL_LOCATION {
QL_ALIAS_CONFIG
index index.html index.htm;
try_files $uri QL_BASE_URLindex.html;
}
location ~ .*\.(html)$ {
add_header Cache-Control no-cache;
}
}
-45
View File
@@ -1,45 +0,0 @@
user root;
worker_processes auto;
pcre_jit on;
error_log /var/log/nginx/error.log warn;
include /etc/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
server_tokens off;
client_max_body_size 4096m;
client_body_buffer_size 20m;
keepalive_timeout 65;
sendfile on;
tcp_nodelay on;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:2m;
gzip on;
gzip_static on;
gzip_types text/plain application/json application/javascript application/x-javascript text/css application/xml text/javascript;
gzip_proxied any;
gzip_vary on;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.0;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
include /etc/nginx/conf.d/*.conf;
}

Some files were not shown because too many files have changed in this diff Show More