Compare commits

...
Author SHA1 Message Date
copilot-swe-agent[bot]andwhyour 0deebcfc88 Hide system log and login log tabs for non-admin users
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-11 16:53:19 +00:00
copilot-swe-agent[bot]andwhyour 07fcb09cc6 Add log isolation and admin-only access for system/login logs
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-11 16:36:12 +00:00
copilot-swe-agent[bot]andwhyour 6aefc61be6 Fix authentication for regular users by validating JWT tokens
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-10 18:10:16 +00:00
copilot-swe-agent[bot]andwhyour bf9be821ba Add ALTER TABLE statements for userId columns in db.ts
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-10 16:57:18 +00:00
copilot-swe-agent[bot]andwhyour d42074f76a Add data migration script and comprehensive migration guide
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-10 16:48:31 +00:00
copilot-swe-agent[bot]andwhyour 5c798a0e93 Add user management frontend interface for admins
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-10 16:38:26 +00:00
whyourandGitHub 15d94f469b Merge branch 'develop' into copilot/enable-multi-user-management 2025-11-10 23:53:10 +08:00
copilot-swe-agent[bot]andwhyour 777fd3fb23 Add user-scoped data filtering for subscription and dependence operations
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-09 16:37:37 +00:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
1f2fd8ac02 Fix URIError from malformed cookies causing white screen on load (#2811)
* Initial plan

* Fix decodeURIComponent error in cookie parsing by adding try-catch

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add type annotation and logging to catch block per code review

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Fix TypeScript errors in deps.ts - remove unused path parameter

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Revert pnpm-lock.yaml to avoid unnecessary lockfile version upgrade

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-09 21:53:06 +08:00
whyour 06aa07329f 修复日志目录逻辑 2025-11-09 21:42:45 +08:00
whyourandGitHub 4cf2858ab0 Merge branch 'develop' into copilot/enable-multi-user-management 2025-11-09 19:50:16 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>whyour
4cb9f57479 环境变量支持置顶 (#2822)
* Initial plan

* Add pin to top feature for environment variables

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Format code with prettier

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add database migration for isPinned column in Envs table

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Use snake_case naming (is_pinned) for database column

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
Co-authored-by: whyour <imwhyour@gmail.com>
2025-11-09 19:43:33 +08:00
CopilotGitHubwhyourcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
c369514741 定时任务支持自定义日志文件或者 /dev/null (#2823)
* Initial plan

* Add log_name field to enable custom log folder naming

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add database migration for log_name column

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Add security validation to prevent path traversal attacks

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Apply prettier formatting to modified files

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Support absolute paths like /dev/null for log redirection

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

* Restrict absolute paths to log directory except /dev/null

Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-09 19:32:40 +08:00
whyour 0e28e1b6c4 修复 pm2 日志目录 2025-11-09 17:52:38 +08:00
copilot-swe-agent[bot]andwhyour 2ff7a186e7 Changes before error encountered
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-07 16:38:20 +00:00
copilot-swe-agent[bot]andwhyour b2b1777c6b Security improvements: Fix ownership checks, add password hashing with bcrypt
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-07 16:37:05 +00:00
copilot-swe-agent[bot]andwhyour f355b4e441 Add user-scoped data filtering for env operations
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-07 16:33:31 +00:00
copilot-swe-agent[bot]andwhyour 489454daa0 Add user-scoped data filtering for cron operations
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-07 16:30:04 +00:00
copilot-swe-agent[bot]andwhyour db93ca9aa9 Add multi-user backend infrastructure: User model, management service and API
Co-authored-by: whyour <22700758+whyour@users.noreply.github.com>
2025-11-07 16:24:00 +00:00
copilot-swe-agent[bot] 4758400df6 Initial plan 2025-11-07 16:12:26 +00:00
whyour 73f8f3c5fa 增加 agent
Create a new agent configuration file for QL agent.
2025-11-08 00:05:56 +08:00
whyour 494ccb3c08 更新 workflow 2025-11-03 00:09:36 +08:00
whyour 399728b433 修复 jwt 认证 2025-11-02 22:28:58 +08:00
whyour 52a1de5063 更新 demo 地址 2025-11-02 21:54:19 +08:00
whyour 18f27a9a69 移除 nginx 2025-11-02 19:29:59 +08:00
whyour 07951964a1 修复模块注入 2025-10-26 22:32:03 +08:00
涛之雨andGitHub a1f888af59 Add validation to dependencies GET endpoint and update service logic (#2778)
* Add validation to dependencies GET endpoint and update service logic

* fix https://github.com/whyour/qinglong/pull/2778/files/6063bc3a67fb329de9b90f7c93524b862bd9eb93#r2266494581

* remove default condition type

* fix query mistakes
2025-10-11 23:23:13 +08:00
涛之雨andGitHub f7472b6e74 Add input validation to script API routes (#2777)
* Add input validation to script API routes

* 优化脚本 API 路由的错误处理逻辑

* Fix optional path compatibility checks

* remove file
2025-10-11 23:20:26 +08:00
whyour a7baeba755 修复 task 命令可能软链失败 2025-10-11 23:15:54 +08:00
涛之雨andGitHub e4f733320d Enable debug backend (#2776) 2025-08-19 11:13:33 +08:00
whyour 55c92dc320 发布版本 v2.19.2 2025-07-12 20:29:49 +08:00
whyour 50769c43dd 修复 command-run 接口日志绑定 2025-07-12 20:29:44 +08:00
whyour 0587644a6b command-run 增加返回 QL-Task-Log 日志路径 2025-06-28 01:05:05 +08:00
whyour 87b934aafe QLAPI.systemNotify 支持自定义通知类型和参数 2025-06-24 02:00:51 +08:00
whyour 7a92e7c6ab 修复取消安装依赖 2025-06-22 21:47:39 +08:00
whyour 1d8403c0ec 修复环境变量过大解析 2025-06-22 21:10:55 +08:00
whyour ef9e38f167 备份数据支持选择模块,支持清除依赖缓存 2025-06-22 14:25:19 +08:00
whyour c9bd053fbd 修改服务启动方式 2025-06-11 00:42:29 +08:00
憶夣andGitHub 57939391b9 ntfy 增加可选的认证与用户动作 (#2741)
* feat:ntfy增加可选的认证

* feat:ntfy增加可选的用户动作

* fix:ntfy动作包含中文报错
2025-06-07 00:26:27 +08:00
whyour 394e96bbf8 修复 health 接口报错 2025-06-07 00:25:47 +08:00
whyour 47c194c1f4 更新版本 v2.19.1 2025-05-24 15:03:08 +08:00
whyour 7d65d96ebd 修复 demo 环境提示 2025-05-24 14:56:49 +08:00
whyour 224000b63b 修复依赖是否安装检查逻辑 2025-05-23 23:45:43 +08:00
whyour 1c18668bad 修复文件下载参数 2025-05-22 00:09:19 +08:00
whyour f94582b68d 修复查询 python 依赖存在逻辑 2025-05-21 01:25:24 +08:00
whyour eb1c00984c 修复任务视图状态包含筛选 2025-05-20 23:40:18 +08:00
whyour 1a185f5682 修复创建脚本可能失败 2025-05-20 01:00:08 +08:00
whyour b6ea8565ec 更新版本 v2.19.0 2025-05-18 22:02:57 +08:00
whyour 2e94d58758 修改启动命令环境变量 2025-05-18 22:02:55 +08:00
whyour 472a3088df 修复启动逻辑 2025-05-17 17:25:50 +08:00
whyour ec3d61a713 修复 /:file 获取日志接口 2025-05-16 00:57:27 +08:00
whyour 95459c33ed 修改 import undici 2025-05-15 01:19:24 +08:00
whyour 5459b72f63 更新 pnpm-lock 2025-05-15 01:15:55 +08:00
whyour 05db2b1df8 修复重置用户名失败 2025-05-15 01:12:01 +08:00
whyour c3072e7712 got 替换为 uudici 2025-05-15 01:01:39 +08:00
whyour 3fafe4d24d 修复依赖强制删除未移除队列 2025-05-13 02:00:37 +08:00
whyour 32bccb3f3e 修复无法识别 python 依赖安装的命令 2025-05-13 00:22:15 +08:00
whyour ac04478d1d 修复重用户名 2025-05-11 18:59:15 +08:00
whyour 8a18baa921 修复登录通知失败造成服务重启 2025-05-11 14:36:16 +08:00
whyour 9a399f8de8 更新 workflow 2025-05-09 23:36:14 +08:00
whyour da639a1f8f 移除部分历史兼容逻辑 2025-05-09 15:37:05 +08:00
whyour 425e49675a 修复删除日志 2025-05-09 00:20:53 +08:00
whyour 8174762c18 demo 环境不自动运行任务 2025-05-08 01:43:14 +08:00
whyour 710a107e73 邮箱通知支持多个收件人 2025-05-08 01:27:25 +08:00
whyour d871585eee 修改服务启动逻辑 2025-05-07 09:30:00 +08:00
whyour 729b405b0f 修复 bootAfter 目录 2025-04-26 23:26:11 +08:00
whyour 71a7c1b9d3 boot 任务改为在依赖安装完成后执行 2025-04-25 23:52:33 +08:00
whyour 40a831f3a2 修复脚本管理查询逻辑 2025-04-25 01:40:37 +08:00
whyour 1befa1bb8c 缓存 node 和 python 依赖 2025-04-23 02:10:39 +08:00
whyour 8009634b44 修复脚本管理增加文件夹 2025-04-19 16:34:28 +08:00
whyour 03a23f6f31 更新 workflow 运行环境 2025-04-19 14:36:21 +08:00
whyour 124e01e93a 修复 QLAPI 修复环境变量 remarks,esm 依赖查不到 2025-04-19 01:38:43 +08:00
whyour 2370923a75 更新版本 v2.18.3 2025-03-19 23:47:53 +08:00
whyour 256e4e7a83 修复任务打开脚本链接 2025-03-19 23:47:51 +08:00
whyour c0ec063333 修复删除脚本参数 2025-03-18 23:17:03 +08:00
whyour 576408de01 脚本管理增加强制打开文件操作 2025-03-16 17:57:13 +08:00
whyour cf94ecfb11 脚本管理和日志管理支持下载 2025-03-13 00:22:24 +08:00
whyour 118c92d9e5 更新版本 v2.18.2 2025-02-28 00:56:43 +08:00
whyour 280085668e 修复未初始化时区设置 2025-02-28 00:56:40 +08:00
whyour bae4073a64 增加重置密码命令 2025-02-27 23:57:26 +08:00
whyour af3e358a6a 系统设置增加时区设置 2025-02-27 00:45:21 +08:00
whyour 64fcbff715 脚本管理增加可预览检查 2025-02-26 01:17:11 +08:00
whyour f9f78b4e05 修改系统内置通知模块名称,避免重复 2025-02-25 00:32:13 +08:00
whyour fa83761d27 修改定时规则类型 2025-02-21 01:35:08 +08:00
whyour 8173075b67 定时任务支持 @once 和 @boot 任务 2025-02-20 02:18:59 +08:00
whyour 496918131f 修复群晖通知参数,任务视图不属于筛选 2025-02-16 12:15:45 +08:00
whyour 5d64aae452 修复 mirror action 2025-02-07 01:56:23 +08:00
whyour 952ea2859e 修复最后运行时长多语言显示 2025-01-31 23:02:25 +08:00
whyour 25e8e8198d 修改 QLAPI 调用校验 2025-01-30 00:55:48 +08:00
whyour 3ab5b0d86b 修复登录失败没有提示 2025-01-29 23:50:46 +08:00
whyour 6df651aa63 更新 ts-proto 版本 2025-01-29 23:29:45 +08:00
152 changed files with 10233 additions and 3215 deletions
+8 -11
View File
@@ -1,14 +1,11 @@
UPDATE_PORT=5300
PUBLIC_PORT=5400
CRON_PORT=5500
BACK_PORT=5600
PORT=5700
GRPC_PORT=5500
BACK_PORT=5700
LOG_LEVEL='debug'
LOG_LEVEL='info'
SECRET='whyour'
JWT_SECRET=
JWT_EXPIRES_IN=
QINIU_AK=''
QINIU_SK=''
QINIU_SCOPE=''
TEMP=''
QINIU_AK=
QINIU_SK=
QINIU_SCOPE=
+4
View File
@@ -0,0 +1,4 @@
---
name: Bug Fixer
description: Fix this issue following our error handling pattern.
---
@@ -14,27 +14,35 @@ on:
workflow_dispatch:
jobs:
to_gitlab:
code_gitlab:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: pixta-dev/repository-mirroring-action@v1
- uses: Yikun/hub-mirror-action@master
with:
target_repo_url: git@gitlab.com:whyour/qinglong.git
ssh_private_key: ${{ secrets.GITLAB_SSH_PK }}
src: github/whyour
dst: gitlab/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITLAB_TOKEN }}
static_list: "qinglong"
force_update: true
to_gitee:
code_gitee:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: pixta-dev/repository-mirroring-action@v1
- uses: Yikun/hub-mirror-action@master
with:
target_repo_url: git@gitee.com:whyour/qinglong.git
ssh_private_key: ${{ secrets.GITLAB_SSH_PK }}
src: github/whyour
dst: gitee/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITEE_TOKEN }}
static_list: "qinglong"
force_update: true
build-static:
runs-on: ubuntu-latest
@@ -70,24 +78,43 @@ jobs:
git config --local user.email 'github-actions[bot]@users.noreply.github.com'
git commit --allow-empty -m "copy static at $(date +'%Y-%m-%d %H:%M:%S')"
git push --force --quiet "https://${{ secrets.API_TOKEN }}@${GITHUB_REPO}.git" ${GITHUB_BRANCH}:${GITHUB_BRANCH}
static_gitlab:
needs: build-static
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: Yikun/hub-mirror-action@master
with:
src: github/whyour
dst: gitlab/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITLAB_TOKEN }}
static_list: "qinglong-static"
force_update: true
mkdir -p ~/.ssh
echo "${PRIVATE_KEY}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
export GIT_SSH_COMMAND="ssh -v -i ~/.ssh/id_rsa -o StrictHostKeyChecking=no -l git"
git remote add gitee "${REPO_GITEE}"
git remote add gitlab "${REPO_GITLAB}"
git gc
git push --force --quiet gitee ${GITHUB_BRANCH}:${GITHUB_BRANCH}
git push --force --quiet gitlab ${GITHUB_BRANCH}:${GITHUB_BRANCH}
static_gitee:
needs: build-static
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: Yikun/hub-mirror-action@master
with:
src: github/whyour
dst: gitee/whyour
dst_key: ${{ secrets.GITLAB_SSH_PK }}
dst_token: ${{ secrets.GITEE_TOKEN }}
static_list: "qinglong-static"
force_update: true
build:
needs: build-static
# 由于 ubuntu-latest 中使用 linux6.x 内核 linux/s390x npm 无法使用
runs-on: ubuntu-20.04
# runs-on: self-hosted
runs-on: ubuntu-22.04
permissions:
packages: write
@@ -156,8 +183,8 @@ jobs:
QL_BRANCH=${{ github.ref_name }}
SOURCE_COMMIT=${{ github.sha }}
network: host
platforms: linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x,linux/386
# platforms: linux/amd64,linux/arm64,linux/ppc64le,linux/s390x,linux/386
# linux/s390x npm 暂不可用
platforms: linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/386
context: .
file: ./docker/Dockerfile
push: true
@@ -169,12 +196,12 @@ jobs:
- name: Image digest
run: |
echo ${{ steps.docker_build.outputs.digest }}
build310:
if: ${{ github.ref_name == 'master' }}
needs: build-static
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
permissions:
packages: write
@@ -222,7 +249,8 @@ jobs:
QL_BRANCH=${{ github.ref_name }}
SOURCE_COMMIT=${{ github.sha }}
network: host
platforms: linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x,linux/386
# linux/s390x npm 暂不可用
platforms: linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/386
context: .
file: ./docker/310.Dockerfile
push: true
+1
View File
@@ -27,3 +27,4 @@ __pycache__
/shell/preload/env.*
/shell/preload/notify.*
/shell/preload/*-notify.json
/shell/preload/__ql_notify__.*
+1 -11
View File
@@ -16,18 +16,8 @@ export default defineConfig({
favicons: [`https://qn.whyour.cn/favicon.svg`],
publicPath: process.env.NODE_ENV === 'production' ? './' : '/',
proxy: {
[`${baseUrl}api/update`]: {
target: 'http://127.0.0.1:5300/',
changeOrigin: true,
pathRewrite: { [`^${baseUrl}api/update`]: '/api' },
},
[`${baseUrl}api/public`]: {
target: 'http://127.0.0.1:5400/',
changeOrigin: true,
pathRewrite: { [`^${baseUrl}api/public`]: '/api' },
},
[`${baseUrl}api`]: {
target: 'http://127.0.0.1:5600/',
target: 'http://127.0.0.1:5700/',
changeOrigin: true,
ws: true,
pathRewrite: { [`^${baseUrl}api`]: '/api' },
+252
View File
@@ -0,0 +1,252 @@
# Multi-User Data Migration Guide
This document explains how to migrate existing data to the multi-user system.
## Overview
When upgrading to the multi-user version of Qinglong, all existing data (cron tasks, environment variables, subscriptions, and dependencies) will be treated as "legacy data" that is accessible to all users.
To properly isolate data between users, you need to migrate existing data to specific user accounts.
## Migration Script
The `migrate-to-multiuser.js` script helps you assign existing legacy data to specific users.
### Prerequisites
- Node.js installed
- Sequelize and dotenv packages (already included in the project)
- At least one user created in the User Management interface
### Usage
#### 1. List All Users
First, see all available users in your system:
```bash
node migrate-to-multiuser.js --list-users
```
Output example:
```
Users in the system:
ID Username Role Status
-- -------- ---- ------
1 admin Admin Enabled
2 user1 User Enabled
3 user2 User Enabled
```
#### 2. Preview Migration (Dry Run)
Before making changes, preview what will be migrated:
```bash
node migrate-to-multiuser.js --userId=1 --dry-run
```
Or by username:
```bash
node migrate-to-multiuser.js --username=admin --dry-run
```
Output example:
```
Legacy Data Statistics:
Cron tasks: 15
Environment variables: 8
Subscriptions: 3
Dependencies: 5
DRY RUN: No changes will be made.
Would assign all legacy data to user ID 1
```
#### 3. Perform Migration
Once you're ready, run the migration without `--dry-run`:
**By User ID:**
```bash
node migrate-to-multiuser.js --userId=1
```
**By Username:**
```bash
node migrate-to-multiuser.js --username=admin
```
Output example:
```
Found user 'admin' with ID 1
Legacy Data Statistics:
Cron tasks: 15
Environment variables: 8
Subscriptions: 3
Dependencies: 5
Migrating data to user ID 1...
✓ Migrated 15 cron tasks
✓ Migrated 8 environment variables
✓ Migrated 3 subscriptions
✓ Migrated 5 dependencies
✓ Migration completed successfully!
```
### Command Line Options
| Option | Description |
|--------|-------------|
| `--userId=<id>` | Assign all legacy data to user with this ID |
| `--username=<name>` | Assign all legacy data to user with this username |
| `--list-users` | List all users in the system |
| `--dry-run` | Show what would be changed without making changes |
| `--help` | Show help message |
## Migration Strategy
### Scenario 1: Single User to Multi-User
If you're upgrading from single-user to multi-user and want to keep all existing data under one admin account:
1. Create an admin user in the User Management interface
2. Run migration: `node migrate-to-multiuser.js --username=admin`
### Scenario 2: Distribute Data to Multiple Users
If you want to distribute existing data to different users:
1. Create all necessary user accounts first
2. Identify which data belongs to which user (you may need to do this manually by checking the database)
3. For each user, manually update the `userId` field in the database tables (`Crontabs`, `Envs`, `Subscriptions`, `Dependences`)
**SQL Example:**
```sql
-- Assign specific cron tasks to user ID 2
UPDATE Crontabs
SET userId = 2
WHERE name LIKE '%user2%' AND userId IS NULL;
-- Assign specific environment variables to user ID 2
UPDATE Envs
SET userId = 2
WHERE name LIKE '%USER2%' AND userId IS NULL;
```
### Scenario 3: Keep as Shared Data
If you want certain data to remain accessible to all users:
- Simply don't run the migration script
- Data with `userId = NULL` remains as "legacy data" accessible to everyone
- This is useful for shared cron tasks or environment variables
## Important Notes
1. **Backup First**: Always backup your database before running migration scripts
```bash
cp data/database.sqlite data/database.sqlite.backup
```
2. **Test in Dry Run**: Always use `--dry-run` first to see what will change
3. **One-Time Operation**: The script only migrates data where `userId` is NULL
- Already migrated data won't be changed
- You can run it multiple times safely
4. **Transaction Safety**: The migration uses database transactions
- If any error occurs, all changes are rolled back
- Your data remains safe
5. **User Must Exist**: The target user must exist before migration
- Create users in the User Management interface first
- Use `--list-users` to verify users exist
## Troubleshooting
### Error: "User not found"
**Problem:** The specified user doesn't exist in the database.
**Solution:**
1. Run `node migrate-to-multiuser.js --list-users` to see available users
2. Create the user in User Management interface if needed
3. Use correct user ID or username
### Error: "Database connection failed"
**Problem:** Cannot connect to the database.
**Solution:**
1. Check that `data/database.sqlite` exists
2. Verify database file permissions
3. Check `QL_DATA_DIR` environment variable if using custom path
### Error: "Migration failed"
**Problem:** An error occurred during migration.
**Solution:**
1. Check the error message for details
2. Verify database is not corrupted
3. Restore from backup if needed
4. Check database file permissions
## Manual Migration
If you prefer to migrate data manually using SQL:
### Connect to Database
```bash
sqlite3 data/database.sqlite
```
### Check Legacy Data
```sql
-- Count legacy cron tasks
SELECT COUNT(*) FROM Crontabs WHERE userId IS NULL;
-- View legacy cron tasks
SELECT id, name, command FROM Crontabs WHERE userId IS NULL;
```
### Migrate Data
```sql
-- Migrate all legacy data to user ID 1
UPDATE Crontabs SET userId = 1 WHERE userId IS NULL;
UPDATE Envs SET userId = 1 WHERE userId IS NULL;
UPDATE Subscriptions SET userId = 1 WHERE userId IS NULL;
UPDATE Dependences SET userId = 1 WHERE userId IS NULL;
```
### Verify Migration
```sql
-- Check if any legacy data remains
SELECT
(SELECT COUNT(*) FROM Crontabs WHERE userId IS NULL) as legacy_crons,
(SELECT COUNT(*) FROM Envs WHERE userId IS NULL) as legacy_envs,
(SELECT COUNT(*) FROM Subscriptions WHERE userId IS NULL) as legacy_subs,
(SELECT COUNT(*) FROM Dependences WHERE userId IS NULL) as legacy_deps;
```
## Support
If you encounter issues with data migration:
1. Check this guide for solutions
2. Review the error messages carefully
3. Ensure you have a recent backup
4. Open an issue on GitHub with:
- Error messages
- Migration command used
- Database statistics (from dry run)
## Related Documentation
- [MULTI_USER_GUIDE.md](./MULTI_USER_GUIDE.md) - Complete multi-user feature guide
- [README.md](./README.md) - Main project documentation
+154
View File
@@ -0,0 +1,154 @@
# 多用户管理功能说明 (Multi-User Management Guide)
## 功能概述 (Overview)
青龙面板现已支持多用户管理和数据隔离功能。管理员可以创建多个用户账号,每个用户只能看到和操作自己的数据。
Qinglong now supports multi-user management with data isolation. Administrators can create multiple user accounts, and each user can only see and operate their own data.
## 用户角色 (User Roles)
### 管理员 (Admin)
- 可以访问所有用户的数据
- 可以创建、编辑、删除用户
- 可以管理系统设置
- Can access all users' data
- Can create, edit, and delete users
- Can manage system settings
### 普通用户 (Regular User)
- 只能访问自己创建的数据
- 可以管理自己的定时任务、环境变量、订阅和依赖
- 无法访问其他用户的数据
- Can only access their own data
- Can manage their own cron jobs, environment variables, subscriptions, and dependencies
- Cannot access other users' data
## API 使用 (API Usage)
### 用户管理接口 (User Management Endpoints)
所有用户管理接口需要管理员权限。
All user management endpoints require admin privileges.
#### 获取用户列表 (Get User List)
```
GET /api/user-management?searchValue=keyword
```
#### 创建用户 (Create User)
```
POST /api/user-management
{
"username": "user1",
"password": "password123",
"role": 1, // 0: admin, 1: user
"status": 0 // 0: active, 1: disabled
}
```
#### 更新用户 (Update User)
```
PUT /api/user-management
{
"id": 1,
"username": "user1",
"password": "newpassword",
"role": 1,
"status": 0
}
```
#### 删除用户 (Delete Users)
```
DELETE /api/user-management
[1, 2, 3] // User IDs to delete
```
## 数据隔离 (Data Isolation)
### 定时任务 (Cron Jobs)
- 每个用户创建的定时任务会自动关联到该用户
- 用户只能查看、编辑、运行、删除自己的定时任务
- 管理员可以查看所有用户的定时任务
### 环境变量 (Environment Variables)
- 每个用户的环境变量相互隔离
- 用户只能查看和修改自己的环境变量
- 管理员可以查看所有环境变量
### 订阅和依赖 (Subscriptions and Dependencies)
- 用户数据完全隔离
- Only accessible by the owning user and admins
## 密码安全 (Password Security)
- 所有密码使用 bcrypt 加密存储
- 密码长度最少为 6 位
- 建议使用强密码
- All passwords are hashed with bcrypt
- Minimum password length is 6 characters
- Strong passwords are recommended
## 数据迁移 (Data Migration)
### 迁移工具 (Migration Tool)
项目提供了数据迁移脚本,可以将现有数据分配给特定用户。
A migration script is provided to assign existing data to specific users.
#### 使用方法 (Usage)
1. **列出所有用户 (List all users)**
```bash
node migrate-to-multiuser.js --list-users
```
2. **预览迁移(不实际执行)(Dry run)**
```bash
node migrate-to-multiuser.js --userId=1 --dry-run
```
3. **将数据迁移到指定用户ID (Migrate to user ID)**
```bash
node migrate-to-multiuser.js --userId=1
```
4. **将数据迁移到指定用户名 (Migrate to username)**
```bash
node migrate-to-multiuser.js --username=admin
```
#### 注意事项 (Important Notes)
- 迁移脚本只会处理 `userId` 为空的数据(遗留数据)
- 已分配给用户的数据不会被修改
- 建议先使用 `--dry-run` 预览变更
- 迁移过程中如果出错会自动回滚
- The script only migrates data where `userId` is NULL (legacy data)
- Data already assigned to users will not be changed
- It's recommended to use `--dry-run` first to preview changes
- Changes are automatically rolled back if an error occurs
## 向后兼容 (Backward Compatibility)
- 原有的单用户系统管理员账号继续有效
- 已存在的数据可以被所有用户访问(遗留数据)
- 新创建的数据会自动关联到创建者
- The original system admin account remains valid
- Existing data is accessible by all users (legacy data)
- Newly created data is automatically associated with the creator
## 注意事项 (Notes)
1. **首次使用**:首次使用多用户功能时,建议先创建一个管理员账号作为备份
2. **密码管理**:请妥善保管用户密码,忘记密码需要管理员重置
3. **数据迁移**:使用提供的 `migrate-to-multiuser.js` 脚本将现有数据分配给特定用户
4. **权限控制**:删除用户不会删除该用户的数据,数据会变为遗留数据
1. **First Use**: When first using multi-user functionality, it's recommended to create an admin account as a backup
2. **Password Management**: Please keep user passwords safe; forgotten passwords need admin reset
3. **Data Migration**: Use the provided `migrate-to-multiuser.js` script to assign existing data to specific users
4. **Permission Control**: Deleting a user doesn't delete their data; the data becomes legacy data
+2 -2
View File
@@ -18,9 +18,9 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
[docker-image-size-image]: https://img.shields.io/docker/image-size/whyour/qinglong?style=flat
[docker-image-size-url]: https://hub.docker.com/r/whyour/qinglong
[Demo](http://demo.ninesix.cc:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[Demo](http://demo.qinglong.online:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[演示](http://demo.ninesix.cc:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
[演示](http://demo.qinglong.online:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
</div>
![cover](https://user-images.githubusercontent.com/22700758/244847235-8dc1ca21-e03f-4606-9458-0541fab60413.png)
+2 -2
View File
@@ -20,9 +20,9 @@ Timed task management platform supporting Python3, JavaScript, Shell, Typescript
[docker-image-size-image]: https://img.shields.io/docker/image-size/whyour/qinglong?style=flat
[docker-image-size-url]: https://hub.docker.com/r/whyour/qinglong
[Demo](http://demo.ninesix.cc:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[Demo](http://demo.qinglong.online:4433/) / [Issues](https://github.com/whyour/qinglong/issues) / [Telegram Channel](https://t.me/jiao_long) / [Buy Me a Coffee](https://www.buymeacoffee.com/qinglong)
[演示](http://demo.ninesix.cc:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
[演示](http://demo.qinglong.online:4433/) / [反馈](https://github.com/whyour/qinglong/issues) / [Telegram 频道](https://t.me/jiao_long) / [打赏开发者](https://user-images.githubusercontent.com/22700758/244744295-29cd0cd1-c8bb-4ea1-adf6-29bd390ad4dd.jpg)
</div>
![cover](https://user-images.githubusercontent.com/22700758/244847235-8dc1ca21-e03f-4606-9458-0541fab60413.png)
Vendored
-9
View File
@@ -1,9 +0,0 @@
import 'express';
declare global {
namespace Express {
interface Request {
platform: string;
}
}
}
+45 -64
View File
@@ -4,7 +4,8 @@ import { Logger } from 'winston';
import CronService from '../services/cron';
import CronViewService from '../services/cronView';
import { celebrate, Joi } from 'celebrate';
import cron_parser from 'cron-parser';
import { commonCronSchema } from '../validation/schedule';
const route = Router();
export default (app: Router) => {
@@ -144,7 +145,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.crontabs(req.query as any);
const data = await cronService.crontabs({
...req.query as any,
userId: req.user?.userId
});
return res.send({ code: 200, data });
} catch (e) {
logger.error('🔥 error: %o', e);
@@ -170,27 +174,17 @@ export default (app: Router) => {
route.post(
'/',
celebrate({
body: Joi.object({
command: Joi.string().required(),
schedule: Joi.string().required(),
name: Joi.string().optional(),
labels: Joi.array().optional(),
sub_id: Joi.number().optional().allow(null),
extra_schedules: Joi.array().optional().allow(null),
task_before: Joi.string().optional().allow('').allow(null),
task_after: Joi.string().optional().allow('').allow(null),
}),
body: Joi.object(commonCronSchema),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
if (cron_parser.parseExpression(req.body.schedule).hasNext()) {
const cronService = Container.get(CronService);
const data = await cronService.create(req.body);
return res.send({ code: 200, data });
} else {
return res.send({ code: 400, message: 'param schedule error' });
}
const cronService = Container.get(CronService);
const data = await cronService.create({
...req.body,
userId: req.user?.userId
});
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
@@ -206,10 +200,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.run(req.body);
const data = await cronService.run(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -223,10 +217,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.stop(req.body);
const data = await cronService.stop(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -246,10 +240,11 @@ export default (app: Router) => {
const data = await cronService.removeLabels(
req.body.ids,
req.body.labels,
req.user?.userId,
);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -266,10 +261,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.addLabels(req.body.ids, req.body.labels);
const data = await cronService.addLabels(req.body.ids, req.body.labels, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -283,10 +278,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.disabled(req.body);
const data = await cronService.disabled(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -300,10 +295,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.enabled(req.body);
const data = await cronService.enabled(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -331,30 +326,16 @@ export default (app: Router) => {
'/',
celebrate({
body: Joi.object({
labels: Joi.array().optional().allow(null),
command: Joi.string().required(),
schedule: Joi.string().required(),
name: Joi.string().optional().allow(null),
sub_id: Joi.number().optional().allow(null),
extra_schedules: Joi.array().optional().allow(null),
task_before: Joi.string().optional().allow('').allow(null),
task_after: Joi.string().optional().allow('').allow(null),
...commonCronSchema,
id: Joi.number().required(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
if (
!req.body.schedule ||
cron_parser.parseExpression(req.body.schedule).hasNext()
) {
const cronService = Container.get(CronService);
const data = await cronService.update(req.body);
return res.send({ code: 200, data });
} else {
return res.send({ code: 400, message: 'param schedule error' });
}
const cronService = Container.get(CronService);
const data = await cronService.update(req.body);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
@@ -370,10 +351,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.remove(req.body);
const data = await cronService.remove(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -387,10 +368,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.pin(req.body);
const data = await cronService.pin(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -404,10 +385,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.unPin(req.body);
const data = await cronService.unPin(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -418,7 +399,7 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const cronService = Container.get(CronService);
const data = await cronService.import_crontab();
const data = await cronService.importCrontab();
return res.send({ code: 200, data });
} catch (e) {
return next(e);
+35 -30
View File
@@ -8,17 +8,28 @@ const route = Router();
export default (app: Router) => {
app.use('/dependencies', route);
route.get('/', async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.dependencies(req.query as any);
return res.send({ code: 200, data });
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
});
route.get(
'/',
celebrate({
query:
Joi.object({
searchValue: Joi.string().optional().allow(''),
type: Joi.string().optional().allow(''),
status: Joi.string().optional().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.dependencies(req.query as any, [], {}, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
},
);
route.post(
'/',
@@ -32,10 +43,9 @@ export default (app: Router) => {
),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.create(req.body);
const data = await dependenceService.create(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
@@ -54,7 +64,6 @@ export default (app: Router) => {
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.update(req.body);
@@ -71,13 +80,12 @@ export default (app: Router) => {
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.remove(req.body);
const data = await dependenceService.remove(req.body, false, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -88,13 +96,12 @@ export default (app: Router) => {
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.remove(req.body, true);
const data = await dependenceService.remove(req.body, true, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -107,7 +114,6 @@ export default (app: Router) => {
}),
}),
async (req: Request<{ id: number }>, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.getDb({ id: req.params.id });
@@ -124,13 +130,12 @@ export default (app: Router) => {
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const dependenceService = Container.get(DependenceService);
const data = await dependenceService.reInstall(req.body);
const data = await dependenceService.reInstall(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -143,10 +148,10 @@ export default (app: Router) => {
async (req: Request, res: Response, next: NextFunction) => {
try {
const dependenceService = Container.get(DependenceService);
await dependenceService.cancel(req.body);
await dependenceService.cancel(req.body, req.user?.userId);
return res.send({ code: 200 });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
+55 -21
View File
@@ -1,12 +1,12 @@
import { Router, Request, Response, NextFunction } from 'express';
import { Container } from 'typedi';
import EnvService from '../services/env';
import { Logger } from 'winston';
import { celebrate, Joi } from 'celebrate';
import multer from 'multer';
import config from '../config';
import { Joi, celebrate } from 'celebrate';
import { NextFunction, Request, Response, Router } from 'express';
import fs from 'fs';
import multer from 'multer';
import { Container } from 'typedi';
import { Logger } from 'winston';
import config from '../config';
import { safeJSONParse } from '../config/util';
import EnvService from '../services/env';
const route = Router();
const storage = multer.diskStorage({
@@ -26,7 +26,7 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.envs(req.query.searchValue as string);
const data = await envService.envs(req.query.searchValue as string, {}, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
logger.error('🔥 error: %o', e);
@@ -54,7 +54,7 @@ export default (app: Router) => {
if (!req.body?.length) {
return res.send({ code: 400, message: '参数不正确' });
}
const data = await envService.create(req.body);
const data = await envService.create(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
@@ -93,10 +93,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.remove(req.body);
const data = await envService.remove(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -132,10 +132,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.disabled(req.body);
const data = await envService.disabled(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -149,10 +149,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.enabled(req.body);
const data = await envService.enabled(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -169,10 +169,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.updateNames(req.body);
const data = await envService.updateNames(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -196,6 +196,40 @@ export default (app: Router) => {
},
);
route.put(
'/pin',
celebrate({
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.pin(req.body);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
},
);
route.put(
'/unpin',
celebrate({
body: Joi.array().items(Joi.number().required()),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
const envService = Container.get(EnvService);
const data = await envService.unPin(req.body);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
}
},
);
route.post(
'/upload',
upload.single('env'),
+27
View File
@@ -0,0 +1,27 @@
import { Router } from 'express';
import Logger from '../loaders/logger';
import { HealthService } from '../services/health';
import Container from 'typedi';
const route = Router();
export default (app: Router) => {
app.use('/', route);
route.get('/health', async (req, res) => {
try {
const healthService = Container.get(HealthService);
const health = await healthService.check();
res.status(200).send({
code: 200,
data: health,
});
} catch (err: any) {
Logger.error('Health check failed:', err);
res.status(500).send({
code: 500,
message: 'Health check failed',
error: err.message,
});
}
});
};
+6
View File
@@ -9,6 +9,9 @@ import open from './open';
import dependence from './dependence';
import system from './system';
import subscription from './subscription';
import update from './update';
import health from './health';
import userManagement from './userManagement';
export default () => {
const app = Router();
@@ -22,6 +25,9 @@ export default () => {
dependence(app);
system(app);
subscription(app);
update(app);
health(app);
userManagement(app);
return app;
};
+217 -23
View File
@@ -1,10 +1,19 @@
import { Router, Request, Response, NextFunction } from 'express';
import { celebrate, Joi } from 'celebrate';
import { NextFunction, Request, Response, Router } from 'express';
import { Container } from 'typedi';
import { Logger } from 'winston';
import config from '../config';
import { getFileContentByName, readDirs, removeAnsi, rmPath } from '../config/util';
import { join, resolve } from 'path';
import { celebrate, Joi } from 'celebrate';
import {
getFileContentByName,
readDirs,
removeAnsi,
rmPath,
IFile,
} from '../config/util';
import LogService from '../services/log';
import CronService from '../services/cron';
import { UserRole } from '../data/user';
import { Crontab } from '../data/cron';
const route = Router();
const blacklist = ['.tmp'];
@@ -15,6 +24,39 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const result = await readDirs(config.logPath, config.logPath, blacklist);
// Filter logs based on user permissions
if (req.user?.role !== UserRole.admin && req.user?.userId) {
const cronService = Container.get(CronService);
const { data: userCrons } = await cronService.crontabs({
searchValue: '',
page: '0',
size: '0',
sorter: '',
filters: '',
queryString: '',
userId: req.user.userId,
});
// Build a set of log paths that the user has access to
const allowedLogPaths = new Set(
userCrons
.filter((cron: Crontab) => cron.log_name && cron.log_name !== '/dev/null')
.map((cron: Crontab) => cron.log_name)
);
// Filter the result to only include logs the user owns
const filteredResult = (result as IFile[]).filter((item: IFile) =>
item.type === 'directory' && (allowedLogPaths.has(item.title) || allowedLogPaths.has(`${item.title}/`))
);
res.send({
code: 200,
data: filteredResult,
});
return;
}
res.send({
code: 200,
data: result,
@@ -29,18 +71,46 @@ export default (app: Router) => {
'/detail',
async (req: Request, res: Response, next: NextFunction) => {
try {
const finalPath = resolve(
config.logPath,
const logService = Container.get(LogService);
const finalPath = logService.checkFilePath(
(req.query.path as string) || '',
(req.query.file as string) || '',
);
if (
blacklist.includes(req.query.path as string) ||
!finalPath.startsWith(config.logPath)
) {
return res.send({ code: 403, message: '暂无权限' });
if (!finalPath || blacklist.includes(req.query.path as string)) {
return res.send({
code: 403,
message: '暂无权限',
});
}
// Check if user has permission to view this log
if (req.user?.role !== UserRole.admin && req.user?.userId) {
const cronService = Container.get(CronService);
const { data: userCrons } = await cronService.crontabs({
searchValue: '',
page: '0',
size: '0',
sorter: '',
filters: '',
queryString: '',
userId: req.user.userId,
});
const logPath = (req.query.path as string) || '';
const hasAccess = userCrons.some((cron: Crontab) =>
cron.log_name &&
cron.log_name !== '/dev/null' &&
(logPath.startsWith(cron.log_name) || cron.log_name.startsWith(logPath))
);
if (!hasAccess) {
return res.send({
code: 403,
message: '暂无权限',
});
}
}
const content = await getFileContentByName(finalPath);
res.send({ code: 200, data: removeAnsi(content) });
} catch (e) {
@@ -53,17 +123,46 @@ export default (app: Router) => {
'/:file',
async (req: Request, res: Response, next: NextFunction) => {
try {
const finalPath = resolve(
config.logPath,
const logService = Container.get(LogService);
const finalPath = logService.checkFilePath(
(req.query.path as string) || '',
(req.params.file as string) || '',
);
if (
blacklist.includes(req.path) ||
!finalPath.startsWith(config.logPath)
) {
return res.send({ code: 403, message: '暂无权限' });
if (!finalPath || blacklist.includes(req.query.path as string)) {
return res.send({
code: 403,
message: '暂无权限',
});
}
// Check if user has permission to view this log
if (req.user?.role !== UserRole.admin && req.user?.userId) {
const cronService = Container.get(CronService);
const { data: userCrons } = await cronService.crontabs({
searchValue: '',
page: '0',
size: '0',
sorter: '',
filters: '',
queryString: '',
userId: req.user.userId,
});
const logPath = (req.query.path as string) || '';
const hasAccess = userCrons.some((cron: Crontab) =>
cron.log_name &&
cron.log_name !== '/dev/null' &&
(logPath.startsWith(cron.log_name) || cron.log_name.startsWith(logPath))
);
if (!hasAccess) {
return res.send({
code: 403,
message: '暂无权限',
});
}
}
const content = await getFileContentByName(finalPath);
res.send({ code: 200, data: content });
} catch (e) {
@@ -83,17 +182,112 @@ export default (app: Router) => {
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path, type } = req.body as {
let { filename, path } = req.body as {
filename: string;
path: string;
type: string;
};
const filePath = join(config.logPath, path, filename);
await rmPath(filePath);
const logService = Container.get(LogService);
const finalPath = logService.checkFilePath(path, filename);
if (!finalPath || blacklist.includes(path)) {
return res.send({
code: 403,
message: '暂无权限',
});
}
// Check if user has permission to delete this log
if (req.user?.role !== UserRole.admin && req.user?.userId) {
const cronService = Container.get(CronService);
const { data: userCrons } = await cronService.crontabs({
searchValue: '',
page: '0',
size: '0',
sorter: '',
filters: '',
queryString: '',
userId: req.user.userId,
});
const hasAccess = userCrons.some((cron: Crontab) =>
cron.log_name &&
cron.log_name !== '/dev/null' &&
(path.startsWith(cron.log_name) || cron.log_name.startsWith(path))
);
if (!hasAccess) {
return res.send({
code: 403,
message: '暂无权限',
});
}
}
await rmPath(finalPath);
res.send({ code: 200 });
} catch (e) {
return next(e);
}
},
);
route.post(
'/download',
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path } = req.body as {
filename: string;
path: string;
};
const logService = Container.get(LogService);
const filePath = logService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
// Check if user has permission to download this log
if (req.user?.role !== UserRole.admin && req.user?.userId) {
const cronService = Container.get(CronService);
const { data: userCrons } = await cronService.crontabs({
searchValue: '',
page: '0',
size: '0',
sorter: '',
filters: '',
queryString: '',
userId: req.user.userId,
});
const hasAccess = userCrons.some((cron: Crontab) =>
cron.log_name &&
cron.log_name !== '/dev/null' &&
(path.startsWith(cron.log_name) || cron.log_name.startsWith(path))
);
if (!hasAccess) {
return res.send({
code: 403,
message: '暂无权限',
});
}
}
return res.download(filePath, filename, (err) => {
if (err) {
return next(err);
}
});
} catch (e) {
return next(e);
}
},
);
};
+123 -66
View File
@@ -1,4 +1,4 @@
import { fileExist, readDirs, readDir, rmPath } from '../config/util';
import { fileExist, readDirs, readDir, rmPath, IFile } from '../config/util';
import { Router, Request, Response, NextFunction } from 'express';
import { Container } from 'typedi';
import { Logger } from 'winston';
@@ -24,55 +24,68 @@ const upload = multer({ storage: storage });
export default (app: Router) => {
app.use('/scripts', route);
route.get('/', async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let result = [];
const blacklist = [
'node_modules',
'.git',
'.pnpm',
'pnpm-lock.yaml',
'yarn.lock',
'package-lock.json',
];
if (req.query.path) {
const targetPath = path.join(
config.scriptPath,
req.query.path as string,
);
result = await readDir(targetPath, config.scriptPath, blacklist);
} else {
result = await readDirs(
config.scriptPath,
config.scriptPath,
blacklist,
(a, b) => {
if (a.type === b.type) {
return a.title.localeCompare(b.title);
} else {
return a.type === 'directory' ? -1 : 1;
}
},
);
route.get(
'/',
celebrate({
query: Joi.object({
path: Joi.string().optional().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let result: IFile[] = [];
const blacklist = [
'node_modules',
'.git',
'.pnpm',
'pnpm-lock.yaml',
'yarn.lock',
'package-lock.json',
];
if (req.query.path) {
result = await readDir(
req.query.path as string,
config.scriptPath,
blacklist,
);
} else {
result = await readDirs(
config.scriptPath,
config.scriptPath,
blacklist,
(a, b) => {
if (a.type === b.type) {
return a.title.localeCompare(b.title);
} else {
return a.type === 'directory' ? -1 : 1;
}
},
);
}
res.send({
code: 200,
data: result,
});
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
res.send({
code: 200,
data: result,
});
} catch (e) {
logger.error('🔥 error: %o', e);
return next(e);
}
});
});
route.get(
'/detail',
celebrate({
query: Joi.object({
path: Joi.string().optional().allow(''),
file: Joi.string().required(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const scriptService = Container.get(ScriptService);
const content = await scriptService.getFile(
req.query.path as string,
req.query?.path as string || '',
req.query.file as string,
);
res.send({ code: 200, data: content });
@@ -84,11 +97,19 @@ export default (app: Router) => {
route.get(
'/:file',
celebrate({
params: Joi.object({
file: Joi.string().required(),
}),
query: Joi.object({
path: Joi.string().optional().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const scriptService = Container.get(ScriptService);
const content = await scriptService.getFile(
req.query.path as string,
req.query?.path as string || '',
req.params.file,
);
res.send({ code: 200, data: content });
@@ -101,8 +122,16 @@ export default (app: Router) => {
route.post(
'/',
upload.single('file'),
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().optional().allow(''),
content: Joi.string().optional().allow(''),
originFilename: Joi.string().optional().allow(''),
directory: Joi.string().optional().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename, path, content, originFilename, directory } =
req.body as {
@@ -124,8 +153,8 @@ export default (app: Router) => {
}
if (config.writePathList.every((x) => !path.startsWith(x))) {
return res.send({
code: 430,
message: '文件路径禁止访问',
code: 403,
message: '暂无权限',
});
}
@@ -175,14 +204,20 @@ export default (app: Router) => {
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename, content, path } = req.body as {
filename: string;
content: string;
path: string;
};
const filePath = join(config.scriptPath, path, filename);
const scriptService = Container.get(ScriptService);
const filePath = scriptService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
await writeFileWithLock(filePath, content);
return res.send({ code: 200 });
} catch (e) {
@@ -196,19 +231,27 @@ export default (app: Router) => {
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().allow(''),
path: Joi.string().optional().allow(''),
type: Joi.string().optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename, path, type } = req.body as {
let { filename, path } = req.body as {
filename: string;
path: string;
type: string;
};
const filePath = join(config.scriptPath, path, filename);
if (!path) {
path = '';
}
const scriptService = Container.get(ScriptService);
const filePath = scriptService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
await rmPath(filePath);
res.send({ code: 200 });
} catch (e) {
@@ -222,24 +265,30 @@ export default (app: Router) => {
celebrate({
body: Joi.object({
filename: Joi.string().required(),
path: Joi.string().optional().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
let { filename } = req.body as {
let { filename, path } = req.body as {
filename: string;
path: string;
};
const filePath = join(config.scriptPath, filename);
// const stats = fs.statSync(filePath);
// res.set({
// 'Content-Type': 'application/octet-stream', //告诉浏览器这是一个二进制文件
// 'Content-Disposition': 'attachment; filename=' + filename, //告诉浏览器这是一个需要下载的文件
// 'Content-Length': stats.size //文件大小
// });
// fs.createReadStream(filePath).pipe(res);
if (!path) {
path = '';
}
const scriptService = Container.get(ScriptService);
const filePath = scriptService.checkFilePath(path, filename);
if (!filePath) {
return res.send({
code: 403,
message: '暂无权限',
});
}
return res.download(filePath, filename, (err) => {
return next(err);
if (err) {
return next(err);
}
});
} catch (e) {
return next(e);
@@ -260,6 +309,9 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
let { filename, content, path } = req.body;
if (!path) {
path = '';
}
const { name, ext } = parse(filename);
const filePath = join(config.scriptPath, path, `${name}.swap${ext}`);
await writeFileWithLock(filePath, content || '');
@@ -285,6 +337,9 @@ export default (app: Router) => {
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path, pid } = req.body;
if (!path) {
path = '';
}
const { name, ext } = parse(filename);
const filePath = join(config.scriptPath, path, `${name}.swap${ext}`);
const logPath = join(config.logPath, path, `${name}.swap`);
@@ -312,12 +367,14 @@ export default (app: Router) => {
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
let { filename, path, type, newFilename } = req.body as {
let { filename, path, newFilename } = req.body as {
filename: string;
path: string;
type: string;
newFilename: string;
};
if (!path) {
path = '';
}
const filePath = join(config.scriptPath, path, filename);
const newPath = join(config.scriptPath, path, newFilename);
await fs.rename(filePath, newPath);
+20 -16
View File
@@ -16,6 +16,7 @@ export default (app: Router) => {
const data = await subscriptionService.list(
req.query.searchValue as string,
req.query.ids as string,
req.user?.userId,
);
return res.send({ code: 200, data });
} catch (e) {
@@ -63,7 +64,10 @@ export default (app: Router) => {
cron_parser.parseExpression(req.body.schedule).hasNext()
) {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.create(req.body);
const data = await subscriptionService.create({
...req.body,
userId: req.user?.userId,
});
return res.send({ code: 200, data });
} else {
return res.send({ code: 400, message: 'param schedule error' });
@@ -83,10 +87,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.run(req.body);
const data = await subscriptionService.run(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -100,10 +104,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.stop(req.body);
const data = await subscriptionService.stop(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -117,10 +121,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.disabled(req.body);
const data = await subscriptionService.disabled(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -134,10 +138,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.enabled(req.body);
const data = await subscriptionService.enabled(req.body, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
@@ -220,10 +224,10 @@ export default (app: Router) => {
const logger: Logger = Container.get('logger');
try {
const subscriptionService = Container.get(SubscriptionService);
const data = await subscriptionService.remove(req.body, req.query);
const data = await subscriptionService.remove(req.body, req.query, req.user?.userId);
return res.send({ code: 200, data });
} catch (e) {
return next(e);
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
+66 -5
View File
@@ -14,6 +14,7 @@ import {
} from '../config/util';
import dayjs from 'dayjs';
import multer from 'multer';
import { UserRole } from '../data/user';
const route = Router();
const storage = multer.diskStorage({
@@ -273,19 +274,20 @@ export default (app: Router) => {
{
onStart: async (cp, startTime) => {
res.setHeader('QL-Task-Pid', `${cp.pid}`);
res.setHeader('QL-Task-Log', `${logPath}`);
},
onEnd: async (cp, endTime, diff) => {
res.end();
},
onError: async (message: string) => {
res.write(`\n${message}`);
res.write(message);
const absolutePath = await handleLogPath(logPath);
await fs.appendFile(absolutePath, `\n${message}`);
await fs.appendFile(absolutePath, message);
},
onLog: async (message: string) => {
res.write(`\n${message}`);
res.write(message);
const absolutePath = await handleLogPath(logPath);
await fs.appendFile(absolutePath, `\n${message}`);
await fs.appendFile(absolutePath, message);
},
},
);
@@ -316,10 +318,15 @@ export default (app: Router) => {
route.put(
'/data/export',
celebrate({
body: Joi.object({
type: Joi.array().items(Joi.string()).optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
await systemService.exportData(res);
await systemService.exportData(res, req.body.type);
} catch (e) {
return next(e);
}
@@ -351,6 +358,14 @@ export default (app: Router) => {
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
// Only admin can view system logs
if (req.user?.role !== UserRole.admin) {
return res.send({
code: 403,
message: '暂无权限',
});
}
const systemService = Container.get(SystemService);
await systemService.getSystemLog(
res,
@@ -369,6 +384,14 @@ export default (app: Router) => {
'/log',
async (req: Request, res: Response, next: NextFunction) => {
try {
// Only admin can delete system logs
if (req.user?.role !== UserRole.admin) {
return res.send({
code: 403,
message: '暂无权限',
});
}
const systemService = Container.get(SystemService);
await systemService.deleteSystemLog();
res.send({ code: 200 });
@@ -384,6 +407,8 @@ export default (app: Router) => {
body: Joi.object({
retries: Joi.number().optional(),
twoFactorActivated: Joi.boolean().optional(),
password: Joi.string().optional(),
username: Joi.string().optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
@@ -396,4 +421,40 @@ export default (app: Router) => {
}
},
);
route.put(
'/config/timezone',
celebrate({
body: Joi.object({
timezone: Joi.string().allow('').allow(null),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.updateTimezone(req.body);
res.send(result);
} catch (e) {
return next(e);
}
},
);
route.put(
'/config/dependence-clean',
celebrate({
body: Joi.object({
type: Joi.string().allow(''),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.cleanDependence(req.body.type);
res.send(result);
} catch (e) {
return next(e);
}
},
);
};
+51
View File
@@ -0,0 +1,51 @@
import { NextFunction, Request, Response, Router } from 'express';
import Container from 'typedi';
import Logger from '../loaders/logger';
import SystemService from '../services/system';
const route = Router();
export default (app: Router) => {
app.use('/update', route);
route.put(
'/reload',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem();
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
route.put(
'/system',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('system');
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
route.put(
'/data',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('data');
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
};
+12 -2
View File
@@ -8,6 +8,8 @@ import path from 'path';
import { v4 as uuidV4 } from 'uuid';
import rateLimit from 'express-rate-limit';
import config from '../config';
import { isDemoEnv } from '../config/util';
import { UserRole } from '../data/user';
const route = Router();
const storage = multer.diskStorage({
@@ -72,9 +74,8 @@ export default (app: Router) => {
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
if (process.env.DeployEnv === 'demo') {
if (isDemoEnv()) {
return res.send({ code: 450, message: '未知错误' });
}
const userService = Container.get(UserService);
@@ -97,6 +98,7 @@ export default (app: Router) => {
username: authInfo.username,
avatar: authInfo.avatar,
twoFactorActivated: authInfo.twoFactorActivated,
role: req.user?.role,
},
});
} catch (e) {
@@ -178,6 +180,14 @@ export default (app: Router) => {
async (req: Request, res: Response, next: NextFunction) => {
const logger: Logger = Container.get('logger');
try {
// Only admin can view login logs
if (req.user?.role !== UserRole.admin) {
return res.send({
code: 403,
message: '暂无权限',
});
}
const userService = Container.get(UserService);
const data = await userService.getLoginLog();
res.send({ code: 200, data });
+114
View File
@@ -0,0 +1,114 @@
import { Router, Request, Response, NextFunction } from 'express';
import { Container } from 'typedi';
import { celebrate, Joi } from 'celebrate';
import UserManagementService from '../services/userManagement';
import { UserRole } from '../data/user';
const route = Router();
// Middleware to check if user is admin
const requireAdmin = (req: Request, res: Response, next: NextFunction) => {
if (req.user && req.user.role === UserRole.admin) {
return next();
}
return res.status(403).send({ code: 403, message: '需要管理员权限' });
};
export default (app: Router) => {
app.use('/user-management', route);
// List all users (admin only)
route.get(
'/',
requireAdmin,
async (req: Request, res: Response, next: NextFunction) => {
try {
const userManagementService = Container.get(UserManagementService);
const data = await userManagementService.list(req.query.searchValue as string);
res.send({ code: 200, data });
} catch (e) {
return next(e);
}
},
);
// Get a specific user (admin only)
route.get(
'/:id',
requireAdmin,
async (req: Request, res: Response, next: NextFunction) => {
try {
const userManagementService = Container.get(UserManagementService);
const data = await userManagementService.get(Number(req.params.id));
res.send({ code: 200, data });
} catch (e) {
return next(e);
}
},
);
// Create a new user (admin only)
route.post(
'/',
requireAdmin,
celebrate({
body: Joi.object({
username: Joi.string().required(),
password: Joi.string().required(),
role: Joi.number().valid(UserRole.admin, UserRole.user).default(UserRole.user),
status: Joi.number().optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const userManagementService = Container.get(UserManagementService);
const data = await userManagementService.create(req.body);
res.send({ code: 200, data, message: '创建用户成功' });
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
// Update a user (admin only)
route.put(
'/',
requireAdmin,
celebrate({
body: Joi.object({
id: Joi.number().required(),
username: Joi.string().required(),
password: Joi.string().required(),
role: Joi.number().valid(UserRole.admin, UserRole.user),
status: Joi.number().optional(),
}),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const userManagementService = Container.get(UserManagementService);
const data = await userManagementService.update(req.body);
res.send({ code: 200, data, message: '更新用户成功' });
} catch (e: any) {
return res.send({ code: 400, message: e.message });
}
},
);
// Delete users (admin only)
route.delete(
'/',
requireAdmin,
celebrate({
body: Joi.array().items(Joi.number()).required(),
}),
async (req: Request, res: Response, next: NextFunction) => {
try {
const userManagementService = Container.get(UserManagementService);
const count = await userManagementService.delete(req.body);
res.send({ code: 200, data: count, message: '删除用户成功' });
} catch (e) {
return next(e);
}
},
);
};
+237 -25
View File
@@ -1,30 +1,242 @@
import 'reflect-metadata'; // We need this in order to use @Decorators
import config from './config';
import 'reflect-metadata';
import cluster, { type Worker } from 'cluster';
import compression from 'compression';
import cors from 'cors';
import express from 'express';
import helmet from 'helmet';
import { Container } from 'typedi';
import config from './config';
import Logger from './loaders/logger';
import { monitoringMiddleware } from './middlewares/monitoring';
import { type GrpcServerService } from './services/grpc';
import { type HttpServerService } from './services/http';
async function startServer() {
const app = express();
await require('./loaders/db').default();
await require('./loaders/initFile').default();
await require('./loaders/app').default({ expressApp: app });
const server = app
.listen(config.port, '0.0.0.0', () => {
Logger.debug(`✌️ 后端服务启动成功!`);
console.debug(`✌️ 后端服务启动成功!`);
process.send?.('ready');
})
.on('error', (err) => {
Logger.error(err);
console.error(err);
process.exit(1);
});
await require('./loaders/server').default({ server });
interface WorkerMetadata {
id: number;
pid: number;
serviceType: string;
startTime: Date;
}
startServer();
class Application {
private app: express.Application;
private httpServerService?: HttpServerService;
private grpcServerService?: GrpcServerService;
private isShuttingDown = false;
private workerMetadataMap = new Map<number, WorkerMetadata>();
constructor() {
this.app = express();
// 创建一个全局中间件,删除查询参数中的t
this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {
if (req.query.t) {
delete req.query.t;
}
next();
});
}
async start() {
try {
if (cluster.isPrimary) {
await this.initializeDatabase();
}
if (cluster.isPrimary) {
this.startMasterProcess();
} else {
await this.startWorkerProcess();
}
} catch (error) {
Logger.error('Failed to start application:', error);
process.exit(1);
}
}
private startMasterProcess() {
this.forkWorker('http');
this.forkWorker('grpc');
cluster.on('exit', (worker, code, signal) => {
const metadata = this.workerMetadataMap.get(worker.id);
if (metadata) {
if (!this.isShuttingDown) {
Logger.error(
`${metadata.serviceType} worker ${worker.process.pid} died (${signal || code
}). Restarting...`,
);
const newWorker = this.forkWorker(metadata.serviceType);
Logger.info(
`Restarted ${metadata.serviceType} worker (New PID: ${newWorker.process.pid})`,
);
}
this.workerMetadataMap.delete(worker.id);
}
});
this.setupMasterShutdown();
}
private forkWorker(serviceType: string): Worker {
const worker = cluster.fork({ SERVICE_TYPE: serviceType });
this.workerMetadataMap.set(worker.id, {
id: worker.id,
pid: worker.process.pid!,
serviceType,
startTime: new Date(),
});
return worker;
}
private async initializeDatabase() {
const dbLoader = await import('./loaders/db');
await dbLoader.default();
}
private setupMiddlewares() {
this.app.use(helmet({
contentSecurityPolicy: false,
}));
this.app.use(cors(config.cors));
this.app.use(compression());
this.app.use(monitoringMiddleware);
}
private setupMasterShutdown() {
const shutdown = async () => {
if (this.isShuttingDown) return;
this.isShuttingDown = true;
const workers = Object.values(cluster.workers || {});
const workerPromises: Promise<void>[] = [];
workers.forEach((worker) => {
if (worker) {
const exitPromise = new Promise<void>((resolve) => {
worker.once('exit', () => {
Logger.info(`Worker ${worker.process.pid} exited`);
resolve();
});
try {
worker.send('shutdown');
} catch (error) {
Logger.warn(
`Failed to send shutdown to worker ${worker.process.pid}:`,
error,
);
}
});
workerPromises.push(exitPromise);
}
});
try {
await Promise.race([
Promise.all(workerPromises),
new Promise<void>((resolve) => {
setTimeout(() => {
Logger.warn('Worker shutdown timeout reached');
resolve();
}, 10000);
}),
]);
process.exit(0);
} catch (error) {
Logger.error('Error during worker shutdown:', error);
process.exit(1);
}
};
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
}
private async startWorkerProcess() {
const serviceType = process.env.SERVICE_TYPE;
if (!serviceType || !['http', 'grpc'].includes(serviceType)) {
Logger.error('Invalid SERVICE_TYPE:', serviceType);
process.exit(1);
}
Logger.info(`✌️ ${serviceType} worker started (PID: ${process.pid})`);
try {
if (serviceType === 'http') {
await this.startHttpService();
} else {
await this.startGrpcService();
}
process.send?.('ready');
} catch (error) {
Logger.error(`${serviceType} worker failed:`, error);
process.exit(1);
}
}
private async startHttpService() {
this.setupMiddlewares();
const { HttpServerService } = await import('./services/http');
this.httpServerService = Container.get(HttpServerService);
const appLoader = await import('./loaders/app');
await appLoader.default({ app: this.app });
const server = await this.httpServerService.initialize(
this.app,
config.port,
);
const serverLoader = await import('./loaders/server');
await (serverLoader.default as any)({ server });
this.setupWorkerShutdown('http');
}
private async startGrpcService() {
const { GrpcServerService } = await import('./services/grpc');
this.grpcServerService = Container.get(GrpcServerService);
await this.grpcServerService.initialize();
this.setupWorkerShutdown('grpc');
}
private setupWorkerShutdown(serviceType: string) {
process.on('message', (msg) => {
if (msg === 'shutdown') {
this.gracefulShutdown(serviceType);
}
});
const shutdown = () => this.gracefulShutdown(serviceType);
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
}
private async gracefulShutdown(serviceType: string) {
if (this.isShuttingDown) return;
this.isShuttingDown = true;
try {
if (serviceType === 'http') {
await this.httpServerService?.shutdown();
} else {
await this.grpcServerService?.shutdown();
}
process.exit(0);
} catch (error) {
Logger.error(`[${serviceType}] Error during shutdown:`, error);
process.exit(1);
}
}
}
const app = new Application();
app.start().catch((error) => {
Logger.error('Application failed to start:', error);
process.exit(1);
});
+26
View File
@@ -23,3 +23,29 @@ export const SAMPLE_FILES = [
target: 'data/scripts/notify.py',
},
];
export const PYTHON_INSTALL_DIR = process.env.PYTHON_HOME;
export const NotificationModeStringMap = {
0: 'gotify',
1: 'goCqHttpBot',
2: 'serverChan',
3: 'pushDeer',
4: 'bark',
5: 'chat',
6: 'telegramBot',
7: 'dingtalkBot',
8: 'weWorkBot',
9: 'weWorkApp',
10: 'aibotk',
11: 'iGot',
12: 'pushPlus',
13: 'wePlusBot',
14: 'email',
15: 'pushMe',
16: 'feishu',
17: 'webhook',
18: 'chronocat',
19: 'ntfy',
20: 'wxPusherBot',
} as const;
+68
View File
@@ -0,0 +1,68 @@
import { request as undiciRequest, Dispatcher } from 'undici';
type RequestBaseOptions = {
dispatcher?: Dispatcher;
json?: Record<string, any>;
form?: string;
headers?: Record<string, string>;
} & Omit<Dispatcher.RequestOptions<null>, 'origin' | 'path' | 'method'>;
type RequestOptionsWithOptions = RequestBaseOptions &
Partial<Pick<Dispatcher.RequestOptions, 'method'>>;
type ResponseTypeMap = {
json: Record<string, any>;
text: string;
};
type ResponseTypeKey = keyof ResponseTypeMap;
async function request(
url: string,
options?: RequestOptionsWithOptions,
): Promise<Dispatcher.ResponseData<null>> {
const { json, form, body, headers = {}, ...rest } = options || {};
const finalHeaders = { ...headers } as Record<string, string>;
let finalBody = body;
if (json) {
finalHeaders['content-type'] = 'application/json';
finalBody = JSON.stringify(json);
} else if (form) {
finalBody = form;
delete finalHeaders['content-type'];
}
const res = await undiciRequest(url, {
method: 'POST',
headers: finalHeaders,
body: finalBody,
...rest,
});
return res;
}
async function post<T extends ResponseTypeKey = 'json'>(
url: string,
options?: RequestBaseOptions & { responseType?: T },
): Promise<ResponseTypeMap[T]> {
const resp = await request(url, { ...options, method: 'POST' });
const rawText = await resp.body.text();
if (options?.responseType === 'text') {
return rawText as ResponseTypeMap[T];
}
try {
return JSON.parse(rawText) as ResponseTypeMap[T];
} catch {
return rawText as ResponseTypeMap[T];
}
}
export const httpClient = {
post,
request,
};
+57 -15
View File
@@ -2,12 +2,60 @@ import dotenv from 'dotenv';
import path from 'path';
import { createRandomString } from './share';
dotenv.config({
path: path.join(__dirname, '../../.env'),
});
interface Config {
port: number;
grpcPort: number;
nodeEnv: string;
isDevelopment: boolean;
isProduction: boolean;
jwt: {
secret: string;
expiresIn?: string;
};
cors: {
origin: string[];
methods: string[];
};
logs: {
level: string;
};
api: {
prefix: string;
};
}
const config: Config = {
port: parseInt(process.env.BACK_PORT || '5700', 10),
grpcPort: parseInt(process.env.GRPC_PORT || '5500', 10),
nodeEnv: process.env.NODE_ENV || 'development',
isDevelopment: process.env.NODE_ENV === 'development',
isProduction: process.env.NODE_ENV === 'production',
logs: {
level: process.env.LOG_LEVEL || 'silly',
},
api: {
prefix: '/api',
},
jwt: {
secret: process.env.JWT_SECRET || 'whyour-secret',
expiresIn: process.env.JWT_EXPIRES_IN,
},
cors: {
origin: process.env.CORS_ORIGIN
? process.env.CORS_ORIGIN.split(',')
: ['*'],
methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
},
};
process.env.NODE_ENV = process.env.NODE_ENV || 'development';
if (!process.env.QL_DIR) {
// 声明QL_DIR环境变量
let qlHomePath = path.join(__dirname, '../../');
// 生产环境
if (qlHomePath.endsWith('/static/')) {
qlHomePath = path.join(qlHomePath, '../');
}
@@ -38,12 +86,13 @@ const dbPath = path.join(dataPath, 'db/');
const uploadPath = path.join(dataPath, 'upload/');
const sshdPath = path.join(dataPath, 'ssh.d/');
const systemLogPath = path.join(dataPath, 'syslog/');
const dependenceCachePath = path.join(dataPath, 'dep_cache/');
const envFile = path.join(preloadPath, 'env.sh');
const jsEnvFile = path.join(preloadPath, 'env.js');
const pyEnvFile = path.join(preloadPath, 'env.py');
const jsNotifyFile = path.join(preloadPath, 'notify.js');
const pyNotifyFile = path.join(preloadPath, 'notify.py');
const jsNotifyFile = path.join(preloadPath, '__ql_notify__.js');
const pyNotifyFile = path.join(preloadPath, '__ql_notify__.py');
const confFile = path.join(configPath, 'config.sh');
const crontabFile = path.join(configPath, 'crontab.list');
const authConfigFile = path.join(configPath, 'auth.json');
@@ -65,17 +114,8 @@ if (envFound.error) {
}
export default {
port: parseInt(process.env.BACK_PORT as string, 10),
cronPort: parseInt(process.env.CRON_PORT as string, 10),
publicPort: parseInt(process.env.PUBLIC_PORT as string, 10),
updatePort: parseInt(process.env.UPDATE_PORT as string, 10),
secret: process.env.SECRET || createRandomString(16, 32),
logs: {
level: process.env.LOG_LEVEL || 'silly',
},
api: {
prefix: '/api',
},
...config,
jwt: config.jwt,
rootPath,
tmpPath,
dataPath,
@@ -118,6 +158,7 @@ export default {
bakPath,
apiWhiteList: [
'/api/user/login',
'/api/health',
'/open/auth/token',
'/api/user/two-factor/login',
'/api/system',
@@ -134,4 +175,5 @@ export default {
sqliteFile,
sshdPath,
systemLogPath,
dependenceCachePath,
};
+126 -99
View File
@@ -1,16 +1,15 @@
import * as fs from 'fs/promises';
import * as path from 'path';
import got from 'got';
import iconv from 'iconv-lite';
import { exec } from 'child_process';
import FormData from 'form-data';
import psTreeFun from 'pstree.remy';
import psTreeFun from 'ps-tree';
import { promisify } from 'util';
import { load } from 'js-yaml';
import config from './index';
import { TASK_COMMAND } from './const';
import { PYTHON_INSTALL_DIR, TASK_COMMAND } from './const';
import Logger from '../loaders/logger';
import { writeFileWithLock } from '../shared/utils';
import { DependenceTypes } from '../data/dependence';
import { FormData } from 'undici';
export * from './share';
@@ -57,78 +56,6 @@ export function getToken(req: any) {
return '';
}
export async function getNetIp(req: any) {
const ipArray = [
...new Set([
...(req.headers['x-real-ip'] || '').split(','),
...(req.headers['x-forwarded-for'] || '').split(','),
req.ip,
...req.ips,
req.socket.remoteAddress,
]),
].filter(Boolean);
let ip = ipArray[0];
if (ipArray.length > 1) {
for (let i = 0; i < ipArray.length; i++) {
const ipNumArray = ipArray[i].split('.');
const tmp = ipNumArray[0] + '.' + ipNumArray[1];
if (
tmp === '192.168' ||
(ipNumArray[0] === '172' &&
ipNumArray[1] >= 16 &&
ipNumArray[1] <= 32) ||
tmp === '10.7' ||
tmp === '127.0'
) {
continue;
}
ip = ipArray[i];
break;
}
}
ip = ip.substr(ip.lastIndexOf(':') + 1, ip.length);
if (ip.includes('127.0') || ip.includes('192.168') || ip.includes('10.7')) {
ip = '';
}
if (!ip) {
return { address: `获取失败`, ip };
}
try {
const csdnApi = got
.get(`https://searchplugin.csdn.net/api/v1/ip/get?ip=${ip}`, {
timeout: 10000,
retry: 0,
})
.text();
const pconlineApi = got
.get(`https://whois.pconline.com.cn/ipJson.jsp?ip=${ip}&json=true`, {
timeout: 10000,
retry: 0,
})
.buffer();
const [csdnBody, pconlineBody] = await await Promise.all<any>([
csdnApi,
pconlineApi,
]);
const csdnRes = JSON.parse(csdnBody);
const pconlineRes = JSON.parse(iconv.decode(pconlineBody, 'GBK'));
let address = '';
if (csdnBody && csdnRes.code == 200) {
address = csdnRes.data.address;
} else if (pconlineRes && pconlineRes.addr) {
address = pconlineRes.addr;
}
return { address, ip };
} catch (error) {
return { address: `获取失败`, ip };
}
}
export function getPlatform(userAgent: string): 'mobile' | 'desktop' {
const ua = userAgent.toLowerCase();
const testUa = (regexp: RegExp) => regexp.test(ua);
@@ -237,7 +164,7 @@ enum FileType {
'file',
}
interface IFile {
export interface IFile {
title: string;
key: string;
type: 'directory' | 'file';
@@ -305,23 +232,51 @@ export async function readDir(
dir: string,
baseDir: string = '',
blacklist: string[] = [],
) {
const relativePath = path.relative(baseDir, dir);
const files = await fs.readdir(dir);
const result: any = files
.filter((x) => !blacklist.includes(x))
.map(async (file: string) => {
const subPath = path.join(dir, file);
): Promise<IFile[]> {
const absoluteDir = path.join(baseDir, dir);
const relativePath = path.relative(baseDir, absoluteDir);
try {
const files = await fs.readdir(absoluteDir);
const result: IFile[] = [];
for (const file of files) {
const subPath = path.join(absoluteDir, file);
const stats = await fs.lstat(subPath);
const key = path.join(relativePath, file);
return {
title: file,
type: stats.isDirectory() ? 'directory' : 'file',
key,
parent: relativePath,
};
});
return result;
if (blacklist.includes(file) || stats.isSymbolicLink()) {
continue;
}
if (stats.isDirectory()) {
result.push({
title: file,
type: 'directory',
key,
parent: relativePath,
createTime: stats.birthtime.getTime(),
children: [],
});
} else {
result.push({
title: file,
type: 'file',
key,
parent: relativePath,
size: stats.size,
createTime: stats.birthtime.getTime(),
});
}
}
return result;
} catch (error: any) {
if (error.code === 'ENOENT') {
return [];
}
throw error;
}
}
export async function promiseExec(command: string): Promise<string> {
@@ -352,9 +307,9 @@ export function parseHeaders(headers: string) {
if (!headers) return {};
const parsed: any = {};
let key;
let val;
let i;
let key: string;
let val: string;
let i: number;
headers &&
headers.split('\n').forEach(function parser(line) {
@@ -433,11 +388,11 @@ export function parseBody(
export function psTree(pid: number): Promise<number[]> {
return new Promise((resolve, reject) => {
psTreeFun(pid, (err: any, pids: number[]) => {
psTreeFun(pid, (err: any, children) => {
if (err) {
reject(err);
}
resolve(pids.filter((x) => !isNaN(x)));
resolve(children.map((x) => Number(x.PID)).filter((x) => !isNaN(x)));
});
});
}
@@ -527,7 +482,7 @@ export function safeJSONParse(value?: string) {
try {
return JSON.parse(value);
} catch (error) {
Logger.error('[JSON.parse失败]', error);
Logger.error('[safeJSONParse失败]', error);
return {};
}
}
@@ -542,3 +497,75 @@ export async function rmPath(path: string) {
Logger.error('[rmPath失败]', error);
}
}
export async function setSystemTimezone(timezone: string): Promise<boolean> {
try {
if (!(await fileExist(`/usr/share/zoneinfo/${timezone}`))) {
throw new Error('Invalid timezone');
}
await promiseExec(`ln -sf /usr/share/zoneinfo/${timezone} /etc/localtime`);
await promiseExec(`echo "${timezone}" > /etc/timezone`);
return true;
} catch (error) {
Logger.error('[setSystemTimezone失败]', error);
return false;
}
}
export function getGetCommand(type: DependenceTypes, name: string): string {
const baseCommands = {
[DependenceTypes.nodejs]: `pnpm ls -g | grep "${name}" | head -1`,
[DependenceTypes.python3]: `
python3 -c "exec('''
name='${name}'
try:
from importlib.metadata import version
print(version(name))
except:
import importlib.util as u
import importlib.metadata as m
spec=u.find_spec(name)
print(name if spec else '')
''')"`,
[DependenceTypes.linux]: `apk info -es ${name}`,
};
return baseCommands[type];
}
export function getInstallCommand(type: DependenceTypes, name: string): string {
const baseCommands = {
[DependenceTypes.nodejs]: 'pnpm add -g',
[DependenceTypes.python3]:
'pip3 install --disable-pip-version-check --root-user-action=ignore',
[DependenceTypes.linux]: 'apk add --no-check-certificate',
};
let command = baseCommands[type];
if (type === DependenceTypes.python3 && PYTHON_INSTALL_DIR) {
command = `${command} --prefix=${PYTHON_INSTALL_DIR}`;
}
return `${command} ${name.trim()}`;
}
export function getUninstallCommand(
type: DependenceTypes,
name: string,
): string {
const baseCommands = {
[DependenceTypes.nodejs]: 'pnpm remove -g',
[DependenceTypes.python3]:
'pip3 uninstall --disable-pip-version-check --root-user-action=ignore -y',
[DependenceTypes.linux]: 'apk del',
};
return `${baseCommands[type]} ${name.trim()}`;
}
export function isDemoEnv() {
return process.env.DeployEnv === 'demo';
}
+7 -1
View File
@@ -21,6 +21,8 @@ export class Crontab {
extra_schedules?: Array<{ schedule: string }>;
task_before?: string;
task_after?: string;
log_name?: string;
userId?: number;
constructor(options: Crontab) {
this.name = options.name;
@@ -45,6 +47,8 @@ export class Crontab {
this.extra_schedules = options.extra_schedules;
this.task_before = options.task_before;
this.task_after = options.task_after;
this.log_name = options.log_name;
this.userId = options.userId;
}
}
@@ -55,7 +59,7 @@ export enum CrontabStatus {
'disabled',
}
export interface CronInstance extends Model<Crontab, Crontab>, Crontab { }
export interface CronInstance extends Model<Crontab, Crontab>, Crontab {}
export const CrontabModel = sequelize.define<CronInstance>('Crontab', {
name: {
unique: 'compositeIndex',
@@ -84,4 +88,6 @@ export const CrontabModel = sequelize.define<CronInstance>('Crontab', {
extra_schedules: DataTypes.JSON,
task_before: DataTypes.STRING,
task_after: DataTypes.STRING,
log_name: DataTypes.STRING,
userId: { type: DataTypes.NUMBER, allowNull: true },
});
+3 -18
View File
@@ -9,6 +9,7 @@ export class Dependence {
name: string;
log?: string[];
remark?: string;
userId?: number;
constructor(options: Dependence) {
this.id = options.id;
@@ -21,6 +22,7 @@ export class Dependence {
this.name = options.name.trim();
this.log = options.log || [];
this.remark = options.remark || '';
this.userId = options.userId;
}
}
@@ -41,30 +43,12 @@ export enum DependenceTypes {
'linux',
}
export enum InstallDependenceCommandTypes {
'pnpm add -g',
'pip3 install --disable-pip-version-check --root-user-action=ignore',
'apk add --no-check-certificate',
}
export enum GetDependenceCommandTypes {
'pnpm ls -g ',
'pip3 show --disable-pip-version-check',
'apk info -es',
}
export enum versionDependenceCommandTypes {
'@',
'==',
'=',
}
export enum unInstallDependenceCommandTypes {
'pnpm remove -g',
'pip3 uninstall --disable-pip-version-check --root-user-action=ignore -y',
'apk del',
}
export interface DependenceInstance
extends Model<Dependence, Dependence>,
Dependence {}
@@ -77,5 +61,6 @@ export const DependenceModel = sequelize.define<DependenceInstance>(
status: DataTypes.NUMBER,
log: DataTypes.JSON,
remark: DataTypes.STRING,
userId: { type: DataTypes.NUMBER, allowNull: true },
},
);
+7 -1
View File
@@ -1,5 +1,5 @@
import { DataTypes, Model } from 'sequelize';
import { sequelize } from '.';
import { DataTypes, Model, ModelDefined } from 'sequelize';
export class Env {
value?: string;
@@ -9,6 +9,8 @@ export class Env {
position?: number;
name?: string;
remarks?: string;
userId?: number;
isPinned?: 1 | 0;
constructor(options: Env) {
this.value = options.value;
@@ -21,6 +23,8 @@ export class Env {
this.position = options.position;
this.name = options.name;
this.remarks = options.remarks || '';
this.userId = options.userId;
this.isPinned = options.isPinned || 0;
}
}
@@ -42,4 +46,6 @@ export const EnvModel = sequelize.define<EnvInstance>('Env', {
position: DataTypes.NUMBER,
name: { type: DataTypes.STRING, unique: 'compositeIndex' },
remarks: DataTypes.STRING,
userId: { type: DataTypes.NUMBER, allowNull: true },
isPinned: DataTypes.NUMBER,
});
+9 -7
View File
@@ -1,5 +1,3 @@
import { IncomingHttpHeaders } from 'http';
export enum NotificationMode {
'gotify' = 'gotify',
'goCqHttpBot' = 'goCqHttpBot',
@@ -49,9 +47,8 @@ export class PushDeerNotification extends NotificationBaseInfo {
public pushDeerUrl = '';
}
export class ChatNotification extends NotificationBaseInfo {
public chatUrl = '';
public chatToken = '';
export class synologyChatNotification extends NotificationBaseInfo {
public synologyChatUrl = '';
}
export class BarkNotification extends NotificationBaseInfo {
@@ -61,7 +58,7 @@ export class BarkNotification extends NotificationBaseInfo {
public barkGroup = 'qinglong';
public barkLevel = 'active';
public barkUrl = '';
public barkArchive=""
public barkArchive = '';
}
export class TelegramBotNotification extends NotificationBaseInfo {
@@ -118,6 +115,7 @@ export class EmailNotification extends NotificationBaseInfo {
public emailService: string = '';
public emailUser: string = '';
public emailPass: string = '';
public emailTo: string = '';
}
export class PushMeNotification extends NotificationBaseInfo {
@@ -150,6 +148,10 @@ export class NtfyNotification extends NotificationBaseInfo {
public ntfyUrl = '';
public ntfyTopic = '';
public ntfyPriority = '';
public ntfyToken = '';
public ntfyUsername = '';
public ntfyPassword = '';
public ntfyActions = '';
}
export class WxPusherBotNotification extends NotificationBaseInfo {
@@ -163,7 +165,7 @@ export interface NotificationInfo
GotifyNotification,
ServerChanNotification,
PushDeerNotification,
ChatNotification,
synologyChatNotification,
BarkNotification,
TelegramBotNotification,
DingtalkBotNotification,
+3
View File
@@ -31,6 +31,7 @@ export class Subscription {
proxy?: string;
autoAddCron?: 1 | 0;
autoDelCron?: 1 | 0;
userId?: number;
constructor(options: Subscription) {
this.id = options.id;
@@ -60,6 +61,7 @@ export class Subscription {
this.proxy = options.proxy;
this.autoAddCron = options.autoAddCron ? 1 : 0;
this.autoDelCron = options.autoDelCron ? 1 : 0;
this.userId = options.userId;
}
}
@@ -111,5 +113,6 @@ export const SubscriptionModel = sequelize.define<SubscriptionInstance>(
proxy: { type: DataTypes.STRING, allowNull: true },
autoAddCron: { type: DataTypes.NUMBER, allowNull: true },
autoDelCron: { type: DataTypes.NUMBER, allowNull: true },
userId: { type: DataTypes.NUMBER, allowNull: true },
},
);
+1
View File
@@ -37,6 +37,7 @@ export interface SystemConfigInfo {
nodeMirror?: string;
pythonMirror?: string;
linuxMirror?: string;
timezone?: string;
}
export interface LoginLogInfo {
+56
View File
@@ -0,0 +1,56 @@
import { sequelize } from '.';
import { DataTypes, Model } from 'sequelize';
export class User {
id?: number;
username: string;
password: string;
role: UserRole;
status: UserStatus;
createdAt?: Date;
updatedAt?: Date;
constructor(options: User) {
this.id = options.id;
this.username = options.username;
this.password = options.password;
this.role = options.role || UserRole.user;
this.status =
typeof options.status === 'number' && UserStatus[options.status]
? options.status
: UserStatus.active;
this.createdAt = options.createdAt;
this.updatedAt = options.updatedAt;
}
}
export enum UserRole {
'admin' = 0,
'user' = 1,
}
export enum UserStatus {
'active' = 0,
'disabled' = 1,
}
export interface UserInstance extends Model<User, User>, User {}
export const UserModel = sequelize.define<UserInstance>('User', {
username: {
type: DataTypes.STRING,
unique: true,
allowNull: false,
},
password: {
type: DataTypes.STRING,
allowNull: false,
},
role: {
type: DataTypes.NUMBER,
defaultValue: UserRole.user,
},
status: {
type: DataTypes.NUMBER,
defaultValue: UserStatus.active,
},
});
-7
View File
@@ -1,7 +0,0 @@
declare namespace Express {
interface Request {
platform: 'desktop' | 'mobile';
}
}
declare module 'pstree.remy';
+13
View File
@@ -0,0 +1,13 @@
export enum ScheduleType {
BOOT = '@boot',
ONCE = '@once',
}
export type ScheduleValidator = (schedule?: string) => boolean;
export type CronSchedulerPayload = {
name: string;
id: string;
schedule: string;
command: string;
extra_schedules: Array<{ schedule: string }>;
};
+9 -10
View File
@@ -5,25 +5,24 @@ import initData from './initData';
import { Application } from 'express';
import linkDeps from './deps';
import initTask from './initTask';
import initFile from './initFile';
export default async ({ expressApp }: { expressApp: Application }) => {
export default async ({ app }: { app: Application }) => {
depInjectorLoader();
Logger.info('✌️ Dependency loaded');
console.log('✌️ Dependency loaded');
await initData();
Logger.info('✌️ Init data loaded');
console.log('✌️ Init data loaded');
await linkDeps();
Logger.info('✌️ Link deps loaded');
console.log('✌️ Link deps loaded');
initFile();
Logger.info('✌️ Init file loaded');
await initData();
Logger.info('✌️ Init data loaded');
initTask();
Logger.info('✌️ Init task loaded');
console.log('✌️ Init task loaded');
expressLoader({ app: expressApp });
expressLoader({ app });
Logger.info('✌️ Express loaded');
console.log('✌️ Express loaded');
};
+8
View File
@@ -0,0 +1,8 @@
import Container from 'typedi';
import CronService from '../services/cron';
export default async () => {
const cronService = Container.get(CronService);
await cronService.bootTask();
};
+25 -3
View File
@@ -6,6 +6,7 @@ import { AppModel } from '../data/open';
import { SystemModel } from '../data/system';
import { SubscriptionModel } from '../data/subscription';
import { CrontabViewModel } from '../data/cronView';
import { UserModel } from '../data/user';
import { sequelize } from '../data';
export default async () => {
@@ -17,6 +18,7 @@ export default async () => {
await EnvModel.sync();
await SubscriptionModel.sync();
await CrontabViewModel.sync();
await UserModel.sync();
// 初始化新增字段
try {
@@ -56,11 +58,31 @@ export default async () => {
try {
await sequelize.query('alter table Crontabs add column task_after TEXT');
} catch (error) {}
try {
await sequelize.query(
'alter table Crontabs add column log_name VARCHAR(255)',
);
} catch (error) {}
try {
await sequelize.query('alter table Envs add column isPinned NUMBER');
} catch (error) {}
// Multi-user support: Add userId columns
try {
await sequelize.query('alter table Crontabs add column userId NUMBER');
} catch (error) {}
try {
await sequelize.query('alter table Envs add column userId NUMBER');
} catch (error) {}
try {
await sequelize.query('alter table Subscriptions add column userId NUMBER');
} catch (error) {}
try {
await sequelize.query('alter table Dependences add column userId NUMBER');
} catch (error) {}
console.log('✌️ DB loaded');
Logger.info('✌️ DB loaded');
} catch (error) {
console.error('✌️ DB load failed');
Logger.error(error);
Logger.error('✌️ DB load failed', error);
}
};
+9 -3
View File
@@ -13,7 +13,7 @@ async function linkToNodeModule(src: string, dst?: string) {
if (!stats) {
await fs.symlink(source, target, 'dir');
}
} catch (error) {}
} catch (error) { }
}
async function linkCommand() {
@@ -36,6 +36,12 @@ async function linkCommand() {
const source = path.join(config.rootPath, 'shell', link.src);
const target = path.join(commandDir, link.dest);
const tmpTarget = path.join(commandDir, link.tmp);
try {
const stats = await fs.lstat(tmpTarget);
if (stats) {
await fs.unlink(tmpTarget);
}
} catch (error) { }
await fs.symlink(source, tmpTarget);
await fs.rename(tmpTarget, target);
}
@@ -52,6 +58,6 @@ export default async (src: string = 'deps') => {
});
watcher
.on('add', (path) => linkToNodeModule(src))
.on('change', (path) => linkToNodeModule(src));
.on('add', () => linkToNodeModule(src))
.on('change', () => linkToNodeModule(src));
};
+39 -19
View File
@@ -7,10 +7,9 @@ import { UnauthorizedError, expressjwt } from 'express-jwt';
import { getPlatform, getToken } from '../config/util';
import rewrite from 'express-urlrewrite';
import { errors } from 'celebrate';
import { createProxyMiddleware } from 'http-proxy-middleware';
import { serveEnv } from '../config/serverEnv';
import Logger from './logger';
import { IKeyvStore, shareStore } from '../shared/store';
import path from 'path';
export default ({ app }: { app: Application }) => {
app.set('trust proxy', 'loopback');
@@ -18,25 +17,18 @@ export default ({ app }: { app: Application }) => {
app.get(`${config.api.prefix}/env.js`, serveEnv);
app.use(`${config.api.prefix}/static`, express.static(config.uploadPath));
app.use(
'/api/public',
createProxyMiddleware({
target: `http://0.0.0.0:${config.publicPort}/api`,
changeOrigin: true,
pathRewrite: { '/api/public': '' },
logger: Logger,
}),
);
app.use(bodyParser.json({ limit: '50mb' }));
app.use(bodyParser.urlencoded({ limit: '50mb', extended: true }));
const frontendPath = path.join(config.rootPath, 'static/dist');
app.use(express.static(frontendPath));
app.use(
expressjwt({
secret: config.secret,
secret: config.jwt.secret,
algorithms: ['HS384'],
}).unless({
path: [...config.apiWhiteList, /^\/open\//],
path: [...config.apiWhiteList, /^\/(?!api\/).*/],
}),
);
@@ -50,7 +42,23 @@ export default ({ app }: { app: Application }) => {
return next();
});
app.use(async (req, res, next) => {
// Extract userId and role from JWT
app.use((req: Request, res, next) => {
if (req.auth) {
const payload = req.auth as any;
req.user = {
userId: payload.userId,
role: payload.role,
};
}
return next();
});
app.use(async (req: Request, res, next) => {
if (!['/open/', '/api/'].some((x) => req.path.startsWith(x))) {
return next();
}
const headerToken = getToken(req);
if (req.path.startsWith('/open/')) {
const apps = await shareStore.getApps();
@@ -80,6 +88,13 @@ export default ({ app }: { app: Application }) => {
return next();
}
// If JWT has been successfully verified by expressjwt middleware, allow the request
// This handles regular users whose tokens are not stored in authInfo
if (req.auth) {
return next();
}
// For system admin, also check against stored token
const authInfo = await shareStore.getAuthInfo();
if (authInfo && headerToken) {
const { token = '', tokens = {} } = authInfo;
@@ -122,10 +137,15 @@ export default ({ app }: { app: Application }) => {
app.use(rewrite('/open/*', '/api/$1'));
app.use(config.api.prefix, routes());
app.use((req, res, next) => {
const err: any = new Error('Not Found');
err['status'] = 404;
next(err);
app.get('*', (_, res, next) => {
const indexPath = path.join(frontendPath, 'index.html');
res.sendFile(indexPath, (err) => {
if (err) {
const err: any = new Error('Not Found');
err['status'] = 404;
next(err);
}
});
});
app.use(errors());
+20 -14
View File
@@ -13,7 +13,7 @@ import { AuthDataType, SystemModel } from '../data/system';
import SystemService from '../services/system';
import UserService from '../services/user';
import { writeFile, readFile } from 'fs/promises';
import { createRandomString, safeJSONParse } from '../config/util';
import { createRandomString, fileExist, safeJSONParse } from '../config/util';
import OpenService from '../services/open';
import { shareStore } from '../shared/store';
import Logger from './logger';
@@ -56,8 +56,11 @@ export default async () => {
password: 'admin',
};
try {
const content = await readFile(config.authConfigFile, 'utf8');
authInfo = safeJSONParse(content);
const authFileExist = await fileExist(config.authConfigFile);
if (authFileExist) {
const content = await readFile(config.authConfigFile, 'utf8');
authInfo = safeJSONParse(content);
}
} catch (error) {
Logger.warn('Failed to read auth config file, using default credentials');
}
@@ -68,24 +71,27 @@ export default async () => {
});
}
const installDependencies = () => {
// 初始化时安装所有处于安装中,安装成功,安装失败的依赖
DependenceModel.findAll({
const installDependencies = async () => {
const docs = await DependenceModel.findAll({
where: {},
order: [
['type', 'DESC'],
['createdAt', 'DESC'],
],
raw: true,
}).then(async (docs) => {
await DependenceModel.update(
{ status: DependenceStatus.queued, log: [] },
{ where: { id: docs.map((x) => x.id!) } },
);
setTimeout(() => {
dependenceService.installDependenceOneByOne(docs);
}, 5000);
});
await DependenceModel.update(
{ status: DependenceStatus.queued, log: [] },
{ where: { id: docs.map((x) => x.id!) } },
);
setTimeout(async () => {
await dependenceService.installDependenceOneByOne(docs);
const bootAfterLoader = await import('./bootAfter');
bootAfterLoader.default();
}, 5000);
};
// 初始化更新 linux/python/nodejs 镜像源配置
+5 -4
View File
@@ -27,8 +27,8 @@ const sampleNotifyJsFile = path.join(samplePath, 'notify.js');
const sampleNotifyPyFile = path.join(samplePath, 'notify.py');
const scriptNotifyJsFile = path.join(scriptPath, 'sendNotify.js');
const scriptNotifyPyFile = path.join(scriptPath, 'notify.py');
const jsNotifyFile = path.join(preloadPath, 'notify.js');
const pyNotifyFile = path.join(preloadPath, 'notify.py');
const jsNotifyFile = path.join(preloadPath, '__ql_notify__.js');
const pyNotifyFile = path.join(preloadPath, '__ql_notify__.py');
const TaskBeforeFile = path.join(configPath, 'task_before.sh');
const TaskBeforeJsFile = path.join(configPath, 'task_before.js');
const TaskBeforePyFile = path.join(configPath, 'task_before.py');
@@ -116,11 +116,12 @@ export default async () => {
`Neither content nor source specified for ${item.target}`,
);
}
const content = item.content || (await fs.readFile(item.source!));
const content =
item.content ||
(await fs.readFile(item.source!, { encoding: 'utf-8' }));
await writeFileWithLock(item.target, content);
}
}
Logger.info('✌️ Init file down');
console.log('✌️ Init file down');
};
+19 -15
View File
@@ -38,21 +38,25 @@ export default async () => {
// 运行删除日志任务
const data = await systemService.getSystemConfig();
if (data && data.info && data.info.logRemoveFrequency) {
const rmlogCron = {
id: data.id as number,
name: '删除日志',
command: `ql rmlog ${data.info.logRemoveFrequency}`,
runOrigin: 'system' as const,
};
await scheduleService.cancelIntervalTask(rmlogCron);
scheduleService.createIntervalTask(
rmlogCron,
{
days: data.info.logRemoveFrequency,
},
true,
);
if (data && data.info) {
if (data.info.logRemoveFrequency) {
const rmlogCron = {
id: data.id as number,
name: '删除日志',
command: `ql rmlog ${data.info.logRemoveFrequency}`,
runOrigin: 'system' as const,
};
await scheduleService.cancelIntervalTask(rmlogCron);
scheduleService.createIntervalTask(
rmlogCron,
{
days: data.info.logRemoveFrequency,
},
true,
);
}
systemService.updateTimezone(data.info);
}
await subscriptionService.setSshConfig();
+50 -25
View File
@@ -4,35 +4,60 @@ import config from '../config';
import path from 'path';
const levelMap: Record<string, string> = {
info: '\ue6f5',
warn: '\ue880',
error: '\ue602',
debug: '\ue67f'
}
const customFormat = winston.format.combine(
winston.format.splat(),
winston.format.timestamp({ format: "YYYY-MM-DD HH:mm:ss" }),
winston.format.align(),
winston.format.printf((i) => `[${levelMap[i.level]}${i.level}] [${[i.timestamp]}]: ${i.message}`),
);
const defaultOptions = {
format: customFormat,
datePattern: "YYYY-MM-DD",
maxSize: "20m",
maxFiles: "7d",
info: '️', // info图标
warn: '⚠️', // 警告图标
error: '❌', // 错误图标
debug: '🐛', // debug调试图标
};
const baseFormat = [
winston.format.splat(),
winston.format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss' }),
winston.format.align(),
];
const consoleFormat = winston.format.combine(
winston.format.colorize({ level: true }),
...baseFormat,
winston.format.printf((info) => {
return `[${info.level} ${info.timestamp}]:${info.message}`;
}),
);
const plainFormat = winston.format.combine(
winston.format.uncolorize(),
...baseFormat,
winston.format.printf((info) => {
return `[${levelMap[info.level] || ''}${info.level} ${info.timestamp}]:${
info.message
}`;
}),
);
const consoleTransport = new winston.transports.Console({
format: consoleFormat,
level: 'debug',
});
const fileTransport = new winston.transports.DailyRotateFile({
filename: path.join(config.systemLogPath, '%DATE%.log'),
datePattern: 'YYYY-MM-DD',
maxSize: '20m',
maxFiles: '7d',
format: plainFormat,
level: config.logs.level || 'info',
});
const LoggerInstance = winston.createLogger({
level: config.logs.level,
level: 'debug',
levels: winston.config.npm.levels,
transports: [
new winston.transports.DailyRotateFile({
filename: path.join(config.systemLogPath, '%DATE%.log'),
...defaultOptions,
})
],
transports: [consoleTransport, fileTransport],
exceptionHandlers: [consoleTransport, fileTransport],
rejectionHandlers: [consoleTransport, fileTransport],
});
LoggerInstance.on('error', (error) => {
console.error('Logger error:', error);
});
export default LoggerInstance;
+36 -14
View File
@@ -4,6 +4,8 @@ import { Container } from 'typedi';
import SockService from '../services/sock';
import { getPlatform } from '../config/util';
import { shareStore } from '../shared/store';
import jwt from 'jsonwebtoken';
import config from '../config';
export default async ({ server }: { server: Server }) => {
const echo = sockJs.createServer({ prefix: '/api/ws', log: () => {} });
@@ -14,26 +16,46 @@ export default async ({ server }: { server: Server }) => {
conn.close('404');
}
const authInfo = await shareStore.getAuthInfo();
const platform = getPlatform(conn.headers['user-agent'] || '') || 'desktop';
const headerToken = conn.url.replace(`${conn.pathname}?token=`, '');
if (authInfo) {
const { token = '', tokens = {} } = authInfo;
if (headerToken === token || tokens[platform] === headerToken) {
sockService.addClient(conn);
let isAuthenticated = false;
conn.on('data', (message) => {
conn.write(message);
});
conn.on('close', function () {
sockService.removeClient(conn);
});
return;
// First try to verify JWT token (for regular users)
if (headerToken) {
try {
jwt.verify(headerToken, config.jwt.secret, { algorithms: ['HS384'] });
isAuthenticated = true;
} catch (error) {
// JWT verification failed, will try authInfo check next
}
}
// Also check against stored token for system admin
if (!isAuthenticated) {
const authInfo = await shareStore.getAuthInfo();
if (authInfo) {
const { token = '', tokens = {} } = authInfo;
if (headerToken === token || tokens[platform] === headerToken) {
isAuthenticated = true;
}
}
}
if (isAuthenticated) {
sockService.addClient(conn);
conn.on('data', (message) => {
conn.write(message);
});
conn.on('close', function () {
sockService.removeClient(conn);
});
return;
}
conn.close('404');
});
-106
View File
@@ -1,106 +0,0 @@
import bodyParser from 'body-parser';
import { errors } from 'celebrate';
import cors from 'cors';
import { Application, NextFunction, Request, Response } from 'express';
import { expressjwt } from 'express-jwt';
import Container from 'typedi';
import config from '../config';
import SystemService from '../services/system';
import Logger from './logger';
export default ({ app }: { app: Application }) => {
app.set('trust proxy', 'loopback');
app.use(cors());
app.use(bodyParser.json({ limit: '50mb' }));
app.use(bodyParser.urlencoded({ limit: '50mb', extended: true }));
app.use(
expressjwt({
secret: config.secret,
algorithms: ['HS384'],
}),
);
app.put(
'/api/reload',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem();
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
app.put(
'/api/system',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('system');
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
app.put(
'/api/data',
async (req: Request, res: Response, next: NextFunction) => {
try {
const systemService = Container.get(SystemService);
const result = await systemService.reloadSystem('data');
res.send(result);
} catch (e) {
Logger.error('🔥 error: %o', e);
return next(e);
}
},
);
app.use((req, res, next) => {
const err: any = new Error('Not Found');
err['status'] = 404;
next(err);
});
app.use(errors());
app.use(
(
err: Error & { status: number },
req: Request,
res: Response,
next: NextFunction,
) => {
if (err.name === 'UnauthorizedError') {
return res
.status(err.status)
.send({ code: 401, message: err.message })
.end();
}
return next(err);
},
);
app.use(
(
err: Error & { status: number },
req: Request,
res: Response,
next: NextFunction,
) => {
res.status(err.status || 500);
res.json({
code: err.status || 500,
message: err.message,
});
},
);
};
+80
View File
@@ -0,0 +1,80 @@
import { Request, Response, NextFunction } from 'express';
import Logger from '../loaders/logger';
import { performance } from 'perf_hooks';
import { metricsService } from '../services/metrics';
interface RequestMetrics {
method: string;
path: string;
duration: number;
statusCode: number;
timestamp: number;
platform?: string;
}
const requestMetrics: RequestMetrics[] = [];
export const monitoringMiddleware = (
req: Request,
res: Response,
next: NextFunction,
) => {
const start = performance.now();
const originalEnd = res.end;
res.end = function (chunk?: any, encoding?: any, cb?: any) {
const duration = performance.now() - start;
const metric: RequestMetrics = {
method: req.method,
path: req.path,
duration,
statusCode: res.statusCode,
timestamp: Date.now(),
platform: req.platform,
};
requestMetrics.push(metric);
metricsService.record('http_request', duration, {
method: req.method,
path: req.path,
statusCode: res.statusCode.toString(),
...(req.platform && { platform: req.platform }),
});
if (requestMetrics.length > 1000) {
requestMetrics.shift();
}
if (duration > 1000) {
Logger.warn(
`Slow request detected: ${req.method} ${
req.path
} took ${duration.toFixed(2)}ms`,
);
}
return originalEnd.call(this, chunk, encoding, cb);
};
next();
};
export const getMetrics = () => {
return {
totalRequests: requestMetrics.length,
averageDuration:
requestMetrics.reduce((acc, curr) => acc + curr.duration, 0) /
requestMetrics.length,
requestsByMethod: requestMetrics.reduce((acc, curr) => {
acc[curr.method] = (acc[curr.method] || 0) + 1;
return acc;
}, {} as Record<string, number>),
requestsByPlatform: requestMetrics.reduce((acc, curr) => {
if (curr.platform) {
acc[curr.platform] = (acc[curr.platform] || 0) + 1;
}
return acc;
}, {} as Record<string, number>),
recentRequests: requestMetrics.slice(-10),
};
};
+186 -1
View File
@@ -8,7 +8,7 @@ message EnvItem {
optional string value = 3;
optional string remarks = 4;
optional int32 status = 5;
optional int32 position = 6;
optional int64 position = 6;
}
message GetEnvsRequest { string searchValue = 1; }
@@ -53,9 +53,190 @@ message Response {
optional string message = 2;
}
message ExtraScheduleItem { string schedule = 1; }
message CronItem {
optional int32 id = 1;
optional string command = 2;
optional string schedule = 3;
optional string name = 4;
repeated string labels = 5;
optional int32 sub_id = 6;
repeated ExtraScheduleItem extra_schedules = 7;
optional string task_before = 8;
optional string task_after = 9;
optional int32 status = 10;
optional string log_path = 11;
optional int32 pid = 12;
optional int64 last_running_time = 13;
optional int64 last_execution_time = 14;
}
message CreateCronRequest {
string command = 1;
string schedule = 2;
optional string name = 3;
repeated string labels = 4;
optional int32 sub_id = 5;
repeated ExtraScheduleItem extra_schedules = 6;
optional string task_before = 7;
optional string task_after = 8;
}
message UpdateCronRequest {
int32 id = 1;
optional string command = 2;
optional string schedule = 3;
optional string name = 4;
repeated string labels = 5;
optional int32 sub_id = 6;
repeated ExtraScheduleItem extra_schedules = 7;
optional string task_before = 8;
optional string task_after = 9;
}
message DeleteCronsRequest { repeated int32 ids = 1; }
message CronsResponse {
int32 code = 1;
repeated CronItem data = 2;
optional string message = 3;
}
message CronResponse {
int32 code = 1;
CronItem data = 2;
optional string message = 3;
}
message CronDetailRequest { string log_path = 1; }
message CronDetailResponse {
int32 code = 1;
CronItem data = 2;
optional string message = 3;
}
enum NotificationMode {
gotify = 0;
goCqHttpBot = 1;
serverChan = 2;
pushDeer = 3;
bark = 4;
chat = 5;
telegramBot = 6;
dingtalkBot = 7;
weWorkBot = 8;
weWorkApp = 9;
aibotk = 10;
iGot = 11;
pushPlus = 12;
wePlusBot = 13;
email = 14;
pushMe = 15;
feishu = 16;
webhook = 17;
chronocat = 18;
ntfy = 19;
wxPusherBot = 20;
}
message NotificationInfo {
NotificationMode type = 1;
optional string gotifyUrl = 2;
optional string gotifyToken = 3;
optional int32 gotifyPriority = 4;
optional string goCqHttpBotUrl = 5;
optional string goCqHttpBotToken = 6;
optional string goCqHttpBotQq = 7;
optional string serverChanKey = 8;
optional string pushDeerKey = 9;
optional string pushDeerUrl = 10;
optional string synologyChatUrl = 11;
optional string barkPush = 12;
optional string barkIcon = 13;
optional string barkSound = 14;
optional string barkGroup = 15;
optional string barkLevel = 16;
optional string barkUrl = 17;
optional string barkArchive = 18;
optional string telegramBotToken = 19;
optional string telegramBotUserId = 20;
optional string telegramBotProxyHost = 21;
optional string telegramBotProxyPort = 22;
optional string telegramBotProxyAuth = 23;
optional string telegramBotApiHost = 24;
optional string dingtalkBotToken = 25;
optional string dingtalkBotSecret = 26;
optional string weWorkBotKey = 27;
optional string weWorkOrigin = 28;
optional string weWorkAppKey = 29;
optional string aibotkKey = 30;
optional string aibotkType = 31;
optional string aibotkName = 32;
optional string iGotPushKey = 33;
optional string pushPlusToken = 34;
optional string pushPlusUser = 35;
optional string pushPlusTemplate = 36;
optional string pushplusChannel = 37;
optional string pushplusWebhook = 38;
optional string pushplusCallbackUrl = 39;
optional string pushplusTo = 40;
optional string wePlusBotToken = 41;
optional string wePlusBotReceiver = 42;
optional string wePlusBotVersion = 43;
optional string emailService = 44;
optional string emailUser = 45;
optional string emailPass = 46;
optional string emailTo = 47;
optional string pushMeKey = 48;
optional string pushMeUrl = 49;
optional string chronocatURL = 50;
optional string chronocatQQ = 51;
optional string chronocatToken = 52;
optional string webhookHeaders = 53;
optional string webhookBody = 54;
optional string webhookUrl = 55;
optional string webhookMethod = 56;
optional string webhookContentType = 57;
optional string larkKey = 58;
optional string ntfyUrl = 59;
optional string ntfyTopic = 60;
optional string ntfyPriority = 61;
optional string ntfyToken = 62;
optional string ntfyUsername = 63;
optional string ntfyPassword = 64;
optional string ntfyActions = 65;
optional string wxPusherBotAppToken = 66;
optional string wxPusherBotTopicIds = 67;
optional string wxPusherBotUids = 68;
}
message SystemNotifyRequest {
string title = 1;
string content = 2;
optional NotificationInfo notificationInfo = 3;
}
service Api {
@@ -69,4 +250,8 @@ service Api {
rpc UpdateEnvNames(UpdateEnvNamesRequest) returns (Response) {}
rpc GetEnvById(GetEnvByIdRequest) returns (EnvResponse) {}
rpc SystemNotify(SystemNotifyRequest) returns (Response) {}
rpc GetCronDetail(CronDetailRequest) returns (CronDetailResponse) {}
rpc CreateCron(CreateCronRequest) returns (CronResponse) {}
rpc UpdateCron(UpdateCronRequest) returns (CronResponse) {}
rpc DeleteCrons(DeleteCronsRequest) returns (Response) {}
}
+2912 -112
View File
File diff suppressed because it is too large Load Diff
+68 -51
View File
@@ -1,10 +1,11 @@
// Code generated by protoc-gen-ts_proto. DO NOT EDIT.
// versions:
// protoc-gen-ts_proto v1.181.2
// protoc-gen-ts_proto v2.6.1
// protoc v3.17.3
// source: back/protos/cron.proto
/* eslint-disable */
import { BinaryReader, BinaryWriter } from "@bufbuild/protobuf/wire";
import {
type CallOptions,
ChannelCredentials,
@@ -17,7 +18,6 @@ import {
type ServiceError,
type UntypedServiceImplementation,
} from "@grpc/grpc-js";
import _m0 from "protobufjs/minimal";
export const protobufPackage = "com.ql.cron";
@@ -29,7 +29,7 @@ export interface ICron {
id: string;
schedule: string;
command: string;
extraSchedules: ISchedule[];
extra_schedules: ISchedule[];
name: string;
}
@@ -51,33 +51,34 @@ function createBaseISchedule(): ISchedule {
return { schedule: "" };
}
export const ISchedule = {
encode(message: ISchedule, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const ISchedule: MessageFns<ISchedule> = {
encode(message: ISchedule, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.schedule !== "") {
writer.uint32(10).string(message.schedule);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): ISchedule {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): ISchedule {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseISchedule();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.schedule = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -105,11 +106,11 @@ export const ISchedule = {
};
function createBaseICron(): ICron {
return { id: "", schedule: "", command: "", extraSchedules: [], name: "" };
return { id: "", schedule: "", command: "", extra_schedules: [], name: "" };
}
export const ICron = {
encode(message: ICron, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const ICron: MessageFns<ICron> = {
encode(message: ICron, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.id !== "") {
writer.uint32(10).string(message.id);
}
@@ -119,8 +120,8 @@ export const ICron = {
if (message.command !== "") {
writer.uint32(26).string(message.command);
}
for (const v of message.extraSchedules) {
ISchedule.encode(v!, writer.uint32(34).fork()).ldelim();
for (const v of message.extra_schedules) {
ISchedule.encode(v!, writer.uint32(34).fork()).join();
}
if (message.name !== "") {
writer.uint32(42).string(message.name);
@@ -128,53 +129,58 @@ export const ICron = {
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): ICron {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): ICron {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseICron();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.id = reader.string();
continue;
case 2:
}
case 2: {
if (tag !== 18) {
break;
}
message.schedule = reader.string();
continue;
case 3:
}
case 3: {
if (tag !== 26) {
break;
}
message.command = reader.string();
continue;
case 4:
}
case 4: {
if (tag !== 34) {
break;
}
message.extraSchedules.push(ISchedule.decode(reader, reader.uint32()));
message.extra_schedules.push(ISchedule.decode(reader, reader.uint32()));
continue;
case 5:
}
case 5: {
if (tag !== 42) {
break;
}
message.name = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -184,8 +190,8 @@ export const ICron = {
id: isSet(object.id) ? globalThis.String(object.id) : "",
schedule: isSet(object.schedule) ? globalThis.String(object.schedule) : "",
command: isSet(object.command) ? globalThis.String(object.command) : "",
extraSchedules: globalThis.Array.isArray(object?.extraSchedules)
? object.extraSchedules.map((e: any) => ISchedule.fromJSON(e))
extra_schedules: globalThis.Array.isArray(object?.extra_schedules)
? object.extra_schedules.map((e: any) => ISchedule.fromJSON(e))
: [],
name: isSet(object.name) ? globalThis.String(object.name) : "",
};
@@ -202,8 +208,8 @@ export const ICron = {
if (message.command !== "") {
obj.command = message.command;
}
if (message.extraSchedules?.length) {
obj.extraSchedules = message.extraSchedules.map((e) => ISchedule.toJSON(e));
if (message.extra_schedules?.length) {
obj.extra_schedules = message.extra_schedules.map((e) => ISchedule.toJSON(e));
}
if (message.name !== "") {
obj.name = message.name;
@@ -219,7 +225,7 @@ export const ICron = {
message.id = object.id ?? "";
message.schedule = object.schedule ?? "";
message.command = object.command ?? "";
message.extraSchedules = object.extraSchedules?.map((e) => ISchedule.fromPartial(e)) || [];
message.extra_schedules = object.extra_schedules?.map((e) => ISchedule.fromPartial(e)) || [];
message.name = object.name ?? "";
return message;
},
@@ -229,33 +235,34 @@ function createBaseAddCronRequest(): AddCronRequest {
return { crons: [] };
}
export const AddCronRequest = {
encode(message: AddCronRequest, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const AddCronRequest: MessageFns<AddCronRequest> = {
encode(message: AddCronRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
for (const v of message.crons) {
ICron.encode(v!, writer.uint32(10).fork()).ldelim();
ICron.encode(v!, writer.uint32(10).fork()).join();
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): AddCronRequest {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): AddCronRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseAddCronRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.crons.push(ICron.decode(reader, reader.uint32()));
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -286,13 +293,13 @@ function createBaseAddCronResponse(): AddCronResponse {
return {};
}
export const AddCronResponse = {
encode(_: AddCronResponse, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const AddCronResponse: MessageFns<AddCronResponse> = {
encode(_: AddCronResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): AddCronResponse {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): AddCronResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseAddCronResponse();
while (reader.pos < end) {
@@ -302,7 +309,7 @@ export const AddCronResponse = {
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -329,33 +336,34 @@ function createBaseDeleteCronRequest(): DeleteCronRequest {
return { ids: [] };
}
export const DeleteCronRequest = {
encode(message: DeleteCronRequest, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const DeleteCronRequest: MessageFns<DeleteCronRequest> = {
encode(message: DeleteCronRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
for (const v of message.ids) {
writer.uint32(10).string(v!);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): DeleteCronRequest {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): DeleteCronRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseDeleteCronRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.ids.push(reader.string());
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -386,13 +394,13 @@ function createBaseDeleteCronResponse(): DeleteCronResponse {
return {};
}
export const DeleteCronResponse = {
encode(_: DeleteCronResponse, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const DeleteCronResponse: MessageFns<DeleteCronResponse> = {
encode(_: DeleteCronResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): DeleteCronResponse {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): DeleteCronResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseDeleteCronResponse();
while (reader.pos < end) {
@@ -402,7 +410,7 @@ export const DeleteCronResponse = {
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -506,3 +514,12 @@ export type Exact<P, I extends P> = P extends Builtin ? P
function isSet(value: any): boolean {
return value !== null && value !== undefined;
}
export interface MessageFns<T> {
encode(message: T, writer?: BinaryWriter): BinaryWriter;
decode(input: BinaryReader | Uint8Array, length?: number): T;
fromJSON(object: any): T;
toJSON(message: T): unknown;
create<I extends Exact<DeepPartial<T>, I>>(base?: I): T;
fromPartial<I extends Exact<DeepPartial<T>, I>>(object: I): T;
}
+27 -16
View File
@@ -1,25 +1,25 @@
// Code generated by protoc-gen-ts_proto. DO NOT EDIT.
// versions:
// protoc-gen-ts_proto v1.181.2
// protoc-gen-ts_proto v2.6.1
// protoc v3.17.3
// source: back/protos/health.proto
/* eslint-disable */
import { BinaryReader, BinaryWriter } from "@bufbuild/protobuf/wire";
import {
type CallOptions,
ChannelCredentials,
Client,
type ClientOptions,
ClientReadableStream,
type ClientReadableStream,
type ClientUnaryCall,
handleServerStreamingCall,
type handleServerStreamingCall,
type handleUnaryCall,
makeGenericClientConstructor,
Metadata,
type ServiceError,
type UntypedServiceImplementation,
} from "@grpc/grpc-js";
import _m0 from "protobufjs/minimal";
export const protobufPackage = "com.ql.health";
@@ -80,33 +80,34 @@ function createBaseHealthCheckRequest(): HealthCheckRequest {
return { service: "" };
}
export const HealthCheckRequest = {
encode(message: HealthCheckRequest, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const HealthCheckRequest: MessageFns<HealthCheckRequest> = {
encode(message: HealthCheckRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.service !== "") {
writer.uint32(10).string(message.service);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): HealthCheckRequest {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): HealthCheckRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseHealthCheckRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 10) {
break;
}
message.service = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -137,33 +138,34 @@ function createBaseHealthCheckResponse(): HealthCheckResponse {
return { status: 0 };
}
export const HealthCheckResponse = {
encode(message: HealthCheckResponse, writer: _m0.Writer = _m0.Writer.create()): _m0.Writer {
export const HealthCheckResponse: MessageFns<HealthCheckResponse> = {
encode(message: HealthCheckResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.status !== 0) {
writer.uint32(8).int32(message.status);
}
return writer;
},
decode(input: _m0.Reader | Uint8Array, length?: number): HealthCheckResponse {
const reader = input instanceof _m0.Reader ? input : _m0.Reader.create(input);
decode(input: BinaryReader | Uint8Array, length?: number): HealthCheckResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
let end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseHealthCheckResponse();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1:
case 1: {
if (tag !== 8) {
break;
}
message.status = reader.int32() as any;
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skipType(tag & 7);
reader.skip(tag & 7);
}
return message;
},
@@ -262,3 +264,12 @@ export type Exact<P, I extends P> = P extends Builtin ? P
function isSet(value: any): boolean {
return value !== null && value !== undefined;
}
export interface MessageFns<T> {
encode(message: T, writer?: BinaryWriter): BinaryWriter;
decode(input: BinaryReader | Uint8Array, length?: number): T;
fromJSON(object: any): T;
toJSON(message: T): unknown;
create<I extends Exact<DeepPartial<T>, I>>(base?: I): T;
fromPartial<I extends Exact<DeepPartial<T>, I>>(object: I): T;
}
-35
View File
@@ -1,35 +0,0 @@
import express from 'express';
import Logger from './loaders/logger';
import config from './config';
import { HealthClient } from './protos/health';
import { credentials } from '@grpc/grpc-js';
const app = express();
const client = new HealthClient(
`0.0.0.0:${config.cronPort}`,
credentials.createInsecure(),
{ 'grpc.enable_http_proxy': 0 },
);
app.get('/api/health', (req, res) => {
client.check({ service: 'cron' }, (err, response) => {
if (err) {
return res.status(200).send({ code: 500, error: err });
}
return res.status(200).send({ code: 200, data: response });
});
});
app
.listen(config.publicPort, '0.0.0.0', async () => {
await require('./loaders/db').default();
Logger.debug(`✌️ 公共服务启动成功!`);
console.debug(`✌️ 公共服务启动成功!`);
process.send?.('ready');
})
.on('error', (err) => {
Logger.error(err);
console.error(err);
process.exit(1);
});
+7 -7
View File
@@ -10,23 +10,23 @@ const addCron = (
callback: sendUnaryData<AddCronResponse>,
) => {
for (const item of call.request.crons) {
const { id, schedule, command, extraSchedules, name } = item;
const { id, schedule, command, extra_schedules, name } = item;
if (scheduleStacks.has(id)) {
scheduleStacks.get(id)?.forEach((x) => x.cancel());
}
Logger.info(
'[schedule][创建定时任务], 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
'[schedule][创建定时任务] 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
id,
name,
schedule,
command,
);
if (extraSchedules?.length) {
extraSchedules.forEach((x) => {
if (extra_schedules?.length) {
extra_schedules.forEach((x) => {
Logger.info(
'[schedule][创建定时任务], 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
'[schedule][创建定时任务] 任务ID: %s, 名称: %s, cron: %s, 执行命令: %s',
id,
name,
x.schedule,
@@ -40,8 +40,8 @@ const addCron = (
Logger.info(`[schedule][准备运行任务] 命令: ${command}`);
runCron(command, item);
}),
...(extraSchedules?.length
? extraSchedules.map((x) =>
...(extra_schedules?.length
? extra_schedules.map((x) =>
nodeSchedule.scheduleJob(id, x.schedule, async () => {
Logger.info(`[schedule][准备运行任务] 命令: ${command}`);
runCron(command, item);
+154 -2
View File
@@ -4,6 +4,7 @@ import EnvService from '../services/env';
import { sendUnaryData, ServerUnaryCall } from '@grpc/grpc-js';
import {
CreateEnvRequest,
CronItem,
DeleteEnvsRequest,
DisableEnvsRequest,
EnableEnvsRequest,
@@ -21,6 +22,16 @@ import {
import LoggerInstance from '../loaders/logger';
import pick from 'lodash/pick';
import SystemService from '../services/system';
import CronService from '../services/cron';
import {
CronDetailRequest,
CronDetailResponse,
CreateCronRequest,
UpdateCronRequest,
DeleteCronsRequest,
CronResponse,
} from '../protos/api';
import { NotificationInfo } from '../data/notify';
Container.set('logger', LoggerInstance);
@@ -62,9 +73,17 @@ export const updateEnv = async (
callback: sendUnaryData<EnvResponse>,
) => {
try {
if (!call.request.env?.id) {
return callback(null, {
code: 400,
data: undefined,
message: 'id parameter is required',
});
}
const envService = Container.get(EnvService);
const data = await envService.update(
pick(call.request.env, ['id', 'name', 'value', 'remark']) as EnvItem,
pick(call.request.env, ['id', 'name', 'value', 'remarks']) as EnvItem,
);
callback(null, { code: 200, data });
} catch (e: any) {
@@ -77,6 +96,13 @@ export const deleteEnvs = async (
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.remove(call.request.ids);
callback(null, { code: 200 });
@@ -90,6 +116,14 @@ export const moveEnv = async (
callback: sendUnaryData<EnvResponse>,
) => {
try {
if (!call.request.id) {
return callback(null, {
code: 400,
data: undefined,
message: 'id parameter is required',
});
}
const envService = Container.get(EnvService);
const data = await envService.move(call.request.id, {
fromIndex: call.request.fromIndex,
@@ -106,6 +140,13 @@ export const disableEnvs = async (
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.disabled(call.request.ids);
callback(null, { code: 200 });
@@ -119,6 +160,13 @@ export const enableEnvs = async (
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.enabled(call.request.ids);
callback(null, { code: 200 });
@@ -132,6 +180,13 @@ export const updateEnvNames = async (
callback: sendUnaryData<Response>,
) => {
try {
if (!call.request.ids || call.request.ids.length === 0) {
return callback(null, {
code: 400,
message: 'ids parameter is required',
});
}
const envService = Container.get(EnvService);
await envService.updateNames({
ids: call.request.ids,
@@ -148,6 +203,14 @@ export const getEnvById = async (
callback: sendUnaryData<EnvResponse>,
) => {
try {
if (!call.request.id) {
return callback(null, {
code: 400,
data: undefined,
message: 'id parameter is required',
});
}
const envService = Container.get(EnvService);
const data = await envService.getDb({ id: call.request.id });
callback(null, {
@@ -165,9 +228,98 @@ export const systemNotify = async (
) => {
try {
const systemService = Container.get(SystemService);
const data = await systemService.notify(call.request);
const data = await systemService.notify({
title: call.request.title,
content: call.request.content,
notificationInfo: call.request.notificationInfo as unknown as NotificationInfo,
});
callback(null, data);
} catch (e: any) {
callback(e);
}
};
const normalizeCronData = (data: CronItem | null): CronItem | undefined => {
if (!data) return undefined;
return {
...data,
sub_id: data.sub_id ?? undefined,
extra_schedules: data.extra_schedules ?? undefined,
pid: data.pid ?? undefined,
task_before: data.task_before ?? undefined,
task_after: data.task_after ?? undefined,
};
};
export const getCronDetail = async (
call: ServerUnaryCall<CronDetailRequest, CronDetailResponse>,
callback: sendUnaryData<CronDetailResponse>,
) => {
try {
if (!call.request.log_path) {
return callback(null, {
code: 400,
data: undefined,
message: 'log_path is required',
});
}
const cronService = Container.get(CronService);
const data = (await cronService.find({
log_path: call.request.log_path,
})) as CronItem;
callback(null, { code: 200, data: normalizeCronData(data) });
} catch (e: any) {
callback(e);
}
};
export const createCron = async (
call: ServerUnaryCall<CreateCronRequest, CronResponse>,
callback: sendUnaryData<CronResponse>,
) => {
try {
const cronService = Container.get(CronService);
const data = (await cronService.create(call.request)) as CronItem;
callback(null, { code: 200, data: normalizeCronData(data) });
} catch (e: any) {
callback(e);
}
};
export const updateCron = async (
call: ServerUnaryCall<UpdateCronRequest, CronResponse>,
callback: sendUnaryData<CronResponse>,
) => {
try {
const cronService = Container.get(CronService);
const { id, ...fields } = call.request;
const updateRequest = {
id,
...Object.entries(fields).reduce((acc: any, [key, value]) => {
if (value !== undefined) {
acc[key] = value;
}
return acc;
}, {}),
} as UpdateCronRequest;
const data = (await cronService.update(updateRequest)) as CronItem;
callback(null, { code: 200, data: normalizeCronData(data) });
} catch (e: any) {
callback(e);
}
};
export const deleteCrons = async (
call: ServerUnaryCall<DeleteCronsRequest, Response>,
callback: sendUnaryData<Response>,
) => {
try {
const cronService = Container.get(CronService);
await cronService.remove(call.request.ids);
callback(null, { code: 200 });
} catch (e: any) {
callback(e);
}
};
+1 -1
View File
@@ -10,7 +10,7 @@ import config from '../config';
class Client {
private client = new CronClient(
`0.0.0.0:${config.cronPort}`,
`0.0.0.0:${config.grpcPort}`,
credentials.createInsecure(),
{ 'grpc.enable_http_proxy': 0 },
);
+1 -1
View File
@@ -10,7 +10,7 @@ const delCron = (
for (const id of call.request.ids) {
if (scheduleStacks.has(id)) {
Logger.info(
'[schedule][取消定时任务], 任务ID: %s',
'[schedule][取消定时任务] 任务ID: %s',
id,
);
scheduleStacks.get(id)?.forEach(x => x.cancel());
-27
View File
@@ -1,27 +0,0 @@
import { Server, ServerCredentials } from '@grpc/grpc-js';
import { CronService } from '../protos/cron';
import { addCron } from './addCron';
import { delCron } from './delCron';
import { HealthService } from '../protos/health';
import { check } from './health';
import config from '../config';
import Logger from '../loaders/logger';
import { ApiService } from '../protos/api';
import * as Api from './api';
const server = new Server({ 'grpc.enable_http_proxy': 0 });
server.addService(HealthService, { check });
server.addService(CronService, { addCron, delCron });
server.addService(ApiService, Api);
server.bindAsync(
`0.0.0.0:${config.cronPort}`,
ServerCredentials.createInsecure(),
(err, port) => {
if (err) {
throw err;
}
Logger.debug(`✌️ 定时服务启动成功!`);
console.debug(`✌️ 定时服务启动成功!`);
process.send?.('ready');
},
);
+3 -3
View File
@@ -3,7 +3,7 @@ import path, { join } from 'path';
import config from '../config';
import { getFileContentByName } from '../config/util';
import { Response } from 'express';
import got from 'got';
import { request } from 'undici';
@Service()
export default class ConfigService {
@@ -27,10 +27,10 @@ export default class ConfigService {
}
if (filePath.startsWith('sample/')) {
const res = await got.get(
const res = await request(
`https://gitlab.com/whyour/qinglong/-/raw/master/${filePath}`,
);
content = res.body;
content = await res.body.text();
} else if (filePath.startsWith('data/scripts/')) {
content = await getFileContentByName(join(config.rootPath, filePath));
} else {
+190 -63
View File
@@ -11,6 +11,7 @@ import {
killTask,
getUniqPath,
safeJSONParse,
isDemoEnv,
} from '../config/util';
import { Op, where, col as colFn, FindOptions, fn, Order } from 'sequelize';
import path from 'path';
@@ -22,10 +23,35 @@ import dayjs from 'dayjs';
import pickBy from 'lodash/pickBy';
import omit from 'lodash/omit';
import { writeFileWithLock } from '../shared/utils';
import { ScheduleType } from '../interface/schedule';
@Service()
export default class CronService {
constructor(@Inject('logger') private logger: winston.Logger) {}
constructor(@Inject('logger') private logger: winston.Logger) { }
private addUserIdFilter(query: any, userId?: number) {
if (userId !== undefined) {
query.userId = userId;
}
return query;
}
private async checkOwnership(ids: number[], userId?: number): Promise<void> {
if (userId === undefined) {
// Admin can access all crons
return;
}
const crons = await CrontabModel.findAll({
where: { id: ids },
attributes: ['id', 'userId'],
});
const unauthorized = crons.filter(
(cron) => cron.userId !== undefined && cron.userId !== userId
);
if (unauthorized.length > 0) {
throw new Error('无权限操作该定时任务');
}
}
private isNodeCron(cron: Crontab) {
const { schedule, extra_schedules } = cron;
@@ -35,22 +61,58 @@ export default class CronService {
return false;
}
private isOnceSchedule(schedule?: string) {
return schedule?.startsWith(ScheduleType.ONCE);
}
private isBootSchedule(schedule?: string) {
return schedule?.startsWith(ScheduleType.BOOT);
}
private isSpecialSchedule(schedule?: string) {
return this.isOnceSchedule(schedule) || this.isBootSchedule(schedule);
}
private async getLogName(cron: Crontab) {
const { log_name, command, id } = cron;
if (log_name === '/dev/null') {
return log_name;
}
let uniqPath = await getUniqPath(command, `${id}`);
if (log_name) {
const normalizedLogName = log_name.startsWith('/') ? log_name : path.join(config.logPath, log_name);
if (normalizedLogName.startsWith(config.logPath)) {
uniqPath = log_name;
}
}
const logDirPath = path.resolve(config.logPath, `${uniqPath}`);
await fs.mkdir(logDirPath, { recursive: true });
return uniqPath;
}
public async create(payload: Crontab): Promise<Crontab> {
const tab = new Crontab(payload);
tab.saved = false;
tab.log_name = await this.getLogName(tab);
const doc = await this.insert(tab);
if (this.isNodeCron(doc)) {
if (isDemoEnv()) {
return doc;
}
if (this.isNodeCron(doc) && !this.isSpecialSchedule(doc.schedule)) {
await cronClient.addCron([
{
name: doc.name || '',
id: String(doc.id),
schedule: doc.schedule!,
command: this.makeCommand(doc),
extraSchedules: doc.extra_schedules || [],
extra_schedules: doc.extra_schedules || [],
},
]);
}
await this.set_crontab();
await this.setCrontab();
return doc;
}
@@ -58,29 +120,34 @@ export default class CronService {
return await CrontabModel.create(payload, { returning: true });
}
public async update(payload: Crontab): Promise<Crontab> {
public async update(payload: Partial<Crontab>): Promise<Crontab> {
const doc = await this.getDb({ id: payload.id });
const tab = new Crontab({ ...doc, ...payload });
tab.saved = false;
tab.log_name = await this.getLogName(tab);
const newDoc = await this.updateDb(tab);
if (doc.isDisabled === 1) {
if (doc.isDisabled === 1 || isDemoEnv()) {
return newDoc;
}
if (this.isNodeCron(doc)) {
await cronClient.delCron([String(doc.id)]);
}
if (this.isNodeCron(newDoc)) {
if (this.isNodeCron(newDoc) && !this.isSpecialSchedule(newDoc.schedule)) {
await cronClient.addCron([
{
name: doc.name || '',
id: String(newDoc.id),
schedule: newDoc.schedule!,
command: this.makeCommand(newDoc),
extraSchedules: newDoc.extra_schedules || [],
extra_schedules: newDoc.extra_schedules || [],
},
]);
}
await this.set_crontab();
await this.setCrontab();
return newDoc;
}
@@ -118,7 +185,7 @@ export default class CronService {
let cron;
try {
cron = await this.getDb({ id });
} catch (err) {}
} catch (err) { }
if (!cron) {
continue;
}
@@ -132,21 +199,26 @@ export default class CronService {
}
}
public async remove(ids: number[]) {
public async remove(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await CrontabModel.destroy({ where: { id: ids } });
await cronClient.delCron(ids.map(String));
await this.set_crontab();
await this.setCrontab();
}
public async pin(ids: number[]) {
public async pin(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await CrontabModel.update({ isPinned: 1 }, { where: { id: ids } });
}
public async unPin(ids: number[]) {
public async unPin(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await CrontabModel.update({ isPinned: 0 }, { where: { id: ids } });
}
public async addLabels(ids: string[], labels: string[]) {
public async addLabels(ids: string[], labels: string[], userId?: number) {
const numIds = ids.map(Number);
await this.checkOwnership(numIds, userId);
const docs = await CrontabModel.findAll({ where: { id: ids } });
for (const doc of docs) {
await CrontabModel.update(
@@ -158,7 +230,9 @@ export default class CronService {
}
}
public async removeLabels(ids: string[], labels: string[]) {
public async removeLabels(ids: string[], labels: string[], userId?: number) {
const numIds = ids.map(Number);
await this.checkOwnership(numIds, userId);
const docs = await CrontabModel.findAll({ where: { id: ids } });
for (const doc of docs) {
await CrontabModel.update(
@@ -179,8 +253,8 @@ export default class CronService {
for (const col of viewQuery.filters) {
const { property, value, operation } = col;
let q: any = {};
let operate2 = null;
let operate = null;
let operate2: any = null;
let operate: any = null;
switch (operation) {
case 'Reg':
operate = Op.like;
@@ -191,21 +265,38 @@ export default class CronService {
operate2 = Op.and;
break;
case 'In':
q[Op.or] = [
{
[property]: Array.isArray(value) ? value : [value],
},
property === 'status' && value.includes(2)
? { isDisabled: 1 }
: {},
];
if (
property === 'status' &&
!value.includes(CrontabStatus.disabled)
) {
q[Op.and] = [
{ [property]: Array.isArray(value) ? value : [value] },
{ isDisabled: 0 },
];
} else {
q[Op.or] = [
{
[property]: Array.isArray(value) ? value : [value],
},
property === 'status' && value.includes(CrontabStatus.disabled)
? { isDisabled: 1 }
: {},
];
}
break;
case 'Nin':
q[Op.and] = [
{
[property]: {
[Op.notIn]: Array.isArray(value) ? value : [value],
},
[Op.or]: [
{
[property]: {
[Op.notIn]: Array.isArray(value) ? value : [value],
},
},
{
[property]: { [Op.is]: null },
},
],
},
property === 'status' && value.includes(2)
? { isDisabled: { [Op.ne]: 1 } }
@@ -320,10 +411,10 @@ export default class CronService {
log_path,
}: {
log_path: string;
}): Promise<Crontab | null> {
}): Promise<Crontab | undefined> {
try {
const result = await CrontabModel.findOne({ where: { log_path } });
return result;
return result?.get({ plain: true });
} catch (error) {
throw error;
}
@@ -336,6 +427,7 @@ export default class CronService {
sorter: string;
filters: string;
queryString: string;
userId?: number;
}): Promise<{ data: Crontab[]; total: number }> {
const searchText = params?.searchValue;
const page = Number(params?.page || '0');
@@ -356,6 +448,7 @@ export default class CronService {
this.formatSearchText(query, searchText);
this.formatFilterQuery(query, filterQuery);
this.formatViewSort(order, viewQuery);
this.addUserIdFilter(query, params?.userId);
if (sorterQuery) {
const { field, type } = sorterQuery;
@@ -374,7 +467,7 @@ export default class CronService {
try {
const result = await CrontabModel.findAll(condition);
const count = await CrontabModel.count({ where: query });
return { data: result, total: count };
return { data: result.map((x) => x.get({ plain: true })), total: count };
} catch (error) {
throw error;
}
@@ -388,7 +481,8 @@ export default class CronService {
return doc.get({ plain: true });
}
public async run(ids: number[]) {
public async run(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await CrontabModel.update(
{ status: CrontabStatus.queued },
{ where: { id: ids } },
@@ -398,7 +492,8 @@ export default class CronService {
});
}
public async stop(ids: number[]) {
public async stop(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
const docs = await CrontabModel.findAll({ where: { id: ids } });
for (const doc of docs) {
if (doc.pid) {
@@ -424,7 +519,7 @@ export default class CronService {
name: cron.name,
command: cron.command,
schedule: cron.schedule,
extraSchedules: cron.extra_schedules,
extra_schedules: cron.extra_schedules,
};
if (cron.status !== CrontabStatus.queued) {
resolve(params);
@@ -435,13 +530,12 @@ export default class CronService {
`[panel][开始执行任务] 参数: ${JSON.stringify(params)}`,
);
let { id, command, log_path } = cron;
const uniqPath = await getUniqPath(command, `${id}`);
let { id, command, log_name } = cron;
const uniqPath = log_name === '/dev/null' ? (await getUniqPath(command, `${id}`)) : log_name;
const logTime = dayjs().format('YYYY-MM-DD-HH-mm-ss-SSS');
const logDirPath = path.resolve(config.logPath, `${uniqPath}`);
if (log_path?.split('/')?.every((x) => x !== uniqPath)) {
await fs.mkdir(logDirPath, { recursive: true });
}
await fs.mkdir(logDirPath, { recursive: true });
const logPath = `${uniqPath}/${logTime}.log`;
const absolutePath = path.resolve(config.logPath, `${logPath}`);
const cp = spawn(
@@ -492,26 +586,32 @@ export default class CronService {
});
}
public async disabled(ids: number[]) {
public async disabled(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await CrontabModel.update({ isDisabled: 1 }, { where: { id: ids } });
await cronClient.delCron(ids.map(String));
await this.set_crontab();
await this.setCrontab();
}
public async enabled(ids: number[]) {
public async enabled(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await CrontabModel.update({ isDisabled: 0 }, { where: { id: ids } });
const docs = await CrontabModel.findAll({ where: { id: ids } });
const sixCron = docs
.filter((x) => this.isNodeCron(x))
.filter((x) => this.isNodeCron(x) && !this.isSpecialSchedule(x.schedule))
.map((doc) => ({
name: doc.name || '',
id: String(doc.id),
schedule: doc.schedule!,
command: this.makeCommand(doc),
extraSchedules: doc.extra_schedules || [],
extra_schedules: doc.extra_schedules || [],
}));
if (isDemoEnv()) {
return;
}
await cronClient.addCron(sixCron);
await this.set_crontab();
await this.setCrontab();
}
public async log(id: number) {
@@ -519,13 +619,18 @@ export default class CronService {
if (!doc) {
return '';
}
if (doc.log_name === '/dev/null') {
return '日志设置为忽略';
}
const absolutePath = path.resolve(config.logPath, `${doc.log_path}`);
const logFileExist = doc.log_path && (await fileExist(absolutePath));
if (logFileExist) {
return await getFileContentByName(`${absolutePath}`);
} else {
return '任务未运行';
return typeof doc.status === 'number' &&
[CrontabStatus.queued, CrontabStatus.running].includes(doc.status)
? '运行中...'
: '日志不存在...';
}
}
@@ -545,7 +650,7 @@ export default class CronService {
files.map(async (x) => ({
filename: x,
directory: relativeDir.replace(config.logPath, ''),
time: (await fs.lstat(`${dir}/${x}`)).mtime.getTime(),
time: (await fs.lstat(`${dir}/${x}`)).birthtimeMs,
})),
)
).sort((a, b) => b.time - a.time);
@@ -559,9 +664,7 @@ export default class CronService {
if (!command.startsWith(TASK_PREFIX) && !command.startsWith(QL_PREFIX)) {
command = `${TASK_PREFIX}${tab.command}`;
}
let commandVariable = `real_time=${Boolean(realTime)} no_tee=true ID=${
tab.id
} `;
let commandVariable = `real_time=${Boolean(realTime)} log_name=${tab.log_name} no_tee=true ID=${tab.id} `;
if (tab.task_before) {
commandVariable += `task_before='${tab.task_before
.replace(/'/g, "'\\''")
@@ -579,15 +682,14 @@ export default class CronService {
return crontab_job_string;
}
private async set_crontab(data?: { data: Crontab[]; total: number }) {
private async setCrontab(data?: { data: Crontab[]; total: number }) {
const tabs = data ?? (await this.crontabs());
var crontab_string = '';
tabs.data.forEach((tab) => {
const _schedule = tab.schedule && tab.schedule.split(/ +/);
if (
tab.isDisabled === 1 ||
_schedule!.length !== 5 ||
tab.extra_schedules?.length
this.isNodeCron(tab) ||
this.isSpecialSchedule(tab.schedule)
) {
crontab_string += '# ';
crontab_string += tab.schedule;
@@ -608,7 +710,7 @@ export default class CronService {
await CrontabModel.update({ saved: true }, { where: {} });
}
public import_crontab() {
public importCrontab() {
exec('crontab -l', (error, stdout, stderr) => {
const lines = stdout.split('\n');
const namePrefix = new Date().getTime();
@@ -644,17 +746,42 @@ export default class CronService {
public async autosave_crontab() {
const tabs = await this.crontabs();
this.set_crontab(tabs);
const sixCron = tabs.data
.filter((x) => this.isNodeCron(x) && x.isDisabled !== 1)
const regularCrons = tabs.data
.filter(
(x) =>
x.isDisabled !== 1 &&
this.isNodeCron(x) &&
!this.isSpecialSchedule(x.schedule),
)
.map((doc) => ({
name: doc.name || '',
id: String(doc.id),
schedule: doc.schedule!,
command: this.makeCommand(doc),
extraSchedules: doc.extra_schedules || [],
extra_schedules: doc.extra_schedules || [],
}));
await cronClient.addCron(sixCron);
if (isDemoEnv()) {
await writeFileWithLock(config.crontabFile, '');
return;
}
await cronClient.addCron(regularCrons);
this.setCrontab(tabs);
}
public async bootTask() {
const tabs = await this.crontabs();
const bootTasks = tabs.data.filter(
(x) => !x.isDisabled && this.isBootSchedule(x.schedule),
);
if (bootTasks.length > 0) {
await CrontabModel.update(
{ status: CrontabStatus.queued },
{ where: { id: bootTasks.map((t) => t.id!) } },
);
for (const task of bootTasks) {
await this.runSingle(task.id!);
}
}
}
}
+70 -48
View File
@@ -3,12 +3,9 @@ import winston from 'winston';
import config from '../config';
import {
Dependence,
InstallDependenceCommandTypes,
DependenceStatus,
DependenceTypes,
unInstallDependenceCommandTypes,
DependenceModel,
GetDependenceCommandTypes,
versionDependenceCommandTypes,
} from '../data/dependence';
import { spawn } from 'cross-spawn';
@@ -19,6 +16,9 @@ import {
getPid,
killTask,
promiseExecSuccess,
getInstallCommand,
getUninstallCommand,
getGetCommand,
} from '../config/util';
import dayjs from 'dayjs';
import taskLimit from '../shared/pLimit';
@@ -28,11 +28,35 @@ export default class DependenceService {
constructor(
@Inject('logger') private logger: winston.Logger,
private sockService: SockService,
) {}
) { }
public async create(payloads: Dependence[]): Promise<Dependence[]> {
private addUserIdFilter(query: any, userId?: number) {
if (userId !== undefined) {
query.userId = userId;
}
return query;
}
private async checkOwnership(ids: number[], userId?: number): Promise<void> {
if (userId === undefined) {
// Admin can access all dependencies
return;
}
const dependencies = await DependenceModel.findAll({
where: { id: ids },
attributes: ['id', 'userId'],
});
const unauthorized = dependencies.filter(
(dep) => dep.userId !== undefined && dep.userId !== userId
);
if (unauthorized.length > 0) {
throw new Error('无权限操作该依赖');
}
}
public async create(payloads: Dependence[], userId?: number): Promise<Dependence[]> {
const tabs = payloads.map((x) => {
const tab = new Dependence({ ...x, status: DependenceStatus.queued });
const tab = new Dependence({ ...x, status: DependenceStatus.queued, userId });
return tab;
});
const docs = await this.insert(tabs);
@@ -65,8 +89,12 @@ export default class DependenceService {
return await this.getDb({ id: payload.id });
}
public async remove(ids: number[], force = false): Promise<Dependence[]> {
public async remove(ids: number[], force = false, userId?: number): Promise<Dependence[]> {
await this.checkOwnership(ids, userId);
const docs = await DependenceModel.findAll({ where: { id: ids } });
for (const doc of docs) {
taskLimit.removeQueuedDependency(doc);
}
const unInstalledDeps = docs.filter(
(x) => x.status !== DependenceStatus.installed,
);
@@ -95,34 +123,34 @@ export default class DependenceService {
searchValue,
type,
status,
}: { searchValue: string; type: string; status: string },
}: {
searchValue: string;
type: keyof typeof DependenceTypes;
status: string;
},
sort: any = [],
query: any = {},
userId?: number,
): Promise<Dependence[]> {
let condition = {
...query,
type: DependenceTypes[type as any],
};
let condition = query;
this.addUserIdFilter(condition, userId);
if (DependenceTypes[type]) {
condition.type = DependenceTypes[type];
}
if (status) {
condition.status = status.split(',').map(Number);
}
if (searchValue) {
const encodeText = encodeURI(searchValue);
const reg = {
condition.name = {
[Op.or]: [
{ [Op.like]: `%${searchValue}%` },
{ [Op.like]: `%${encodeText}%` },
],
};
condition = {
...condition,
name: reg,
};
}
try {
const result = await this.find(condition, sort);
return result as any;
return await this.find(condition, sort);
} catch (error) {
throw error;
}
@@ -132,34 +160,39 @@ export default class DependenceService {
docs: Dependence[],
isInstall: boolean = true,
force: boolean = false,
) {
): Promise<void> {
docs.forEach((dep) => {
this.installOrUninstallDependency(dep, isInstall, force);
});
return taskLimit.waitDependencyQueueDone();
}
public async reInstall(ids: number[]): Promise<Dependence[]> {
public async reInstall(ids: number[], userId?: number): Promise<Dependence[]> {
await this.checkOwnership(ids, userId);
await DependenceModel.update(
{ status: DependenceStatus.queued, log: [] },
{ where: { id: ids } },
);
const docs = await DependenceModel.findAll({ where: { id: ids } });
for (const doc of docs) {
taskLimit.removeQueuedDependency(doc);
}
this.installDependenceOneByOne(docs, true, true);
return docs;
}
public async cancel(ids: number[]) {
public async cancel(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
const docs = await DependenceModel.findAll({ where: { id: ids } });
for (const doc of docs) {
taskLimit.removeQueuedDependency(doc);
const depInstallCommand = InstallDependenceCommandTypes[doc.type];
const depUnInstallCommand = unInstallDependenceCommandTypes[doc.type];
const installCmd = `${depInstallCommand} ${doc.name.trim()}`;
const unInstallCmd = `${depUnInstallCommand} ${doc.name.trim()}`;
const depInstallCommand = getInstallCommand(doc.type, doc.name);
const depUnInstallCommand = getUninstallCommand(doc.type, doc.name);
const pids = await Promise.all([
getPid(installCmd),
getPid(unInstallCmd),
getPid(depInstallCommand),
getPid(depUnInstallCommand),
]);
for (const pid of pids) {
pid && (await killTask(pid));
@@ -226,11 +259,9 @@ export default class DependenceService {
? 'installDependence'
: 'uninstallDependence';
let depName = dependency.name.trim();
const depRunCommand = (
isInstall
? InstallDependenceCommandTypes
: unInstallDependenceCommandTypes
)[dependency.type];
const command = isInstall
? getInstallCommand(dependency.type, depName)
: getUninstallCommand(dependency.type, depName);
const actionText = isInstall ? '安装' : '删除';
const startTime = dayjs();
@@ -246,7 +277,7 @@ export default class DependenceService {
// 判断是否已经安装过依赖
if (isInstall && !force) {
const getCommandPrefix = GetDependenceCommandTypes[dependency.type];
const getCommand = getGetCommand(dependency.type, depName);
const depVersionStr = versionDependenceCommandTypes[dependency.type];
let depVersion = '';
if (depName.includes(depVersionStr)) {
@@ -263,13 +294,7 @@ export default class DependenceService {
const isLinuxDependence = dependency.type === DependenceTypes.linux;
const isPythonDependence =
dependency.type === DependenceTypes.python3;
const depInfo = (
await promiseExecSuccess(
isNodeDependence
? `${getCommandPrefix} | grep "${depName}" | head -1`
: `${getCommandPrefix} ${depName}`,
)
)
const depInfo = (await promiseExecSuccess(getCommand))
.replace(/\s{2,}/, ' ')
.replace(/\s+$/, '');
@@ -304,12 +329,9 @@ export default class DependenceService {
const proxyStr = dependenceProxyFileExist
? `source ${config.dependenceProxyFile} &&`
: '';
const cp = spawn(
`${proxyStr} ${depRunCommand} ${dependency.name.trim()}`,
{
shell: '/bin/bash',
},
);
const cp = spawn(`${proxyStr} ${command}`, {
shell: '/bin/bash',
});
cp.stdout.on('data', async (data) => {
this.sockService.sendMessage({
+49 -12
View File
@@ -1,7 +1,8 @@
import { Service, Inject } from 'typedi';
import groupBy from 'lodash/groupBy';
import { FindOptions, Op } from 'sequelize';
import { Inject, Service } from 'typedi';
import winston from 'winston';
import config from '../config';
import * as fs from 'fs/promises';
import {
Env,
EnvModel,
@@ -11,16 +12,38 @@ import {
minPosition,
stepPosition,
} from '../data/env';
import groupBy from 'lodash/groupBy';
import { FindOptions, Op } from 'sequelize';
import { writeFileWithLock } from '../shared/utils';
@Service()
export default class EnvService {
constructor(@Inject('logger') private logger: winston.Logger) {}
public async create(payloads: Env[]): Promise<Env[]> {
const envs = await this.envs();
private addUserIdFilter(query: any, userId?: number) {
if (userId !== undefined) {
query.userId = userId;
}
return query;
}
private async checkOwnership(ids: number[], userId?: number): Promise<void> {
if (userId === undefined) {
// Admin can access all envs
return;
}
const envs = await EnvModel.findAll({
where: { id: ids },
attributes: ['id', 'userId'],
});
const unauthorized = envs.filter(
(env) => env.userId !== undefined && env.userId !== userId
);
if (unauthorized.length > 0) {
throw new Error('无权限操作该环境变量');
}
}
public async create(payloads: Env[], userId?: number): Promise<Env[]> {
const envs = await this.envs('', {}, userId);
let position = initPosition;
if (
envs &&
@@ -31,7 +54,7 @@ export default class EnvService {
}
const tabs = payloads.map((x) => {
position = position - stepPosition;
const tab = new Env({ ...x, position });
const tab = new Env({ ...x, position, userId });
return tab;
});
const docs = await this.insert(tabs);
@@ -62,7 +85,8 @@ export default class EnvService {
return await this.getDb({ id: payload.id });
}
public async remove(ids: number[]) {
public async remove(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await EnvModel.destroy({ where: { id: ids } });
await this.set_envs();
}
@@ -119,8 +143,9 @@ export default class EnvService {
return parseFloat(position.toPrecision(16));
}
public async envs(searchText: string = '', query: any = {}): Promise<Env[]> {
public async envs(searchText: string = '', query: any = {}, userId?: number): Promise<Env[]> {
let condition = { ...query };
this.addUserIdFilter(condition, userId);
if (searchText) {
const encodeText = encodeURI(searchText);
const reg = {
@@ -147,6 +172,7 @@ export default class EnvService {
}
try {
const result = await this.find(condition, [
['isPinned', 'DESC'],
['position', 'DESC'],
['createdAt', 'ASC'],
]);
@@ -172,7 +198,8 @@ export default class EnvService {
return doc.get({ plain: true });
}
public async disabled(ids: number[]) {
public async disabled(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await EnvModel.update(
{ status: EnvStatus.disabled },
{ where: { id: ids } },
@@ -180,16 +207,26 @@ export default class EnvService {
await this.set_envs();
}
public async enabled(ids: number[]) {
public async enabled(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await EnvModel.update({ status: EnvStatus.normal }, { where: { id: ids } });
await this.set_envs();
}
public async updateNames({ ids, name }: { ids: number[]; name: string }) {
public async updateNames({ ids, name }: { ids: number[]; name: string }, userId?: number) {
await this.checkOwnership(ids, userId);
await EnvModel.update({ name }, { where: { id: ids } });
await this.set_envs();
}
public async pin(ids: number[]) {
await EnvModel.update({ isPinned: 1 }, { where: { id: ids } });
}
public async unPin(ids: number[]) {
await EnvModel.update({ isPinned: 0 }, { where: { id: ids } });
}
public async set_envs() {
const envs = await this.envs('', {
name: { [Op.not]: null },
+64
View File
@@ -0,0 +1,64 @@
import { Server, ServerCredentials } from '@grpc/grpc-js';
import { CronService } from '../protos/cron';
import { HealthService } from '../protos/health';
import { ApiService } from '../protos/api';
import { addCron } from '../schedule/addCron';
import { delCron } from '../schedule/delCron';
import { check } from '../schedule/health';
import * as Api from '../schedule/api';
import Logger from '../loaders/logger';
import { promisify } from 'util';
import config from '../config';
import { metricsService } from './metrics';
import { Service } from 'typedi';
@Service()
export class GrpcServerService {
private server: Server = new Server({ 'grpc.enable_http_proxy': 0 });
async initialize() {
try {
this.server.addService(HealthService, { check });
this.server.addService(CronService, { addCron, delCron });
this.server.addService(ApiService, Api);
const grpcPort = config.grpcPort;
const bindAsync = promisify(this.server.bindAsync).bind(this.server);
await bindAsync(
`0.0.0.0:${grpcPort}`,
ServerCredentials.createInsecure(),
);
Logger.debug(`✌️ gRPC service started successfully`);
metricsService.record('grpc_service_start', 1, {
port: grpcPort.toString(),
});
return grpcPort;
} catch (err) {
Logger.error('Failed to start gRPC service:', err);
throw err;
}
}
async shutdown() {
try {
if (this.server) {
await new Promise((resolve) => {
this.server.tryShutdown(() => {
Logger.debug('gRPC service stopped');
metricsService.record('grpc_service_stop', 1);
resolve(null);
});
});
}
} catch (err) {
Logger.error('Error while shutting down gRPC service:', err);
throw err;
}
}
getServer() {
return this.server;
}
}
+72
View File
@@ -0,0 +1,72 @@
import { Service } from 'typedi';
import Logger from '../loaders/logger';
import { GrpcServerService } from './grpc';
import { HttpServerService } from './http';
interface HealthStatus {
status: 'ok' | 'error';
services: {
http: boolean;
grpc: boolean;
};
metrics: {
uptime: number;
memory: {
used: number;
total: number;
};
};
}
@Service()
export class HealthService {
private startTime = Date.now();
constructor(
private grpcServerService: GrpcServerService,
private httpServerService: HttpServerService,
) {}
async check(): Promise<HealthStatus> {
const status: HealthStatus = {
status: 'ok',
services: {
http: true,
grpc: true,
},
metrics: {
uptime: Math.floor((Date.now() - this.startTime) / 1000),
memory: {
used: process.memoryUsage().heapUsed,
total: process.memoryUsage().heapTotal,
},
},
};
try {
const httpServer = this.httpServerService.getServer();
if (!httpServer) {
status.services.http = false;
status.status = 'error';
}
} catch (err) {
status.services.http = false;
status.status = 'error';
Logger.error('HTTP server check failed:', err);
}
try {
const grpcServer = this.grpcServerService.getServer();
if (!grpcServer) {
status.services.grpc = false;
status.status = 'error';
}
} catch (err) {
status.services.grpc = false;
status.status = 'error';
Logger.error('gRPC server check failed:', err);
}
return status;
}
}
+53
View File
@@ -0,0 +1,53 @@
import express from 'express';
import Logger from '../loaders/logger';
import { metricsService } from './metrics';
import { Service } from 'typedi';
import { Server } from 'http';
@Service()
export class HttpServerService {
private server?: Server = undefined;
async initialize(expressApp: express.Application, port: number) {
try {
return new Promise((resolve, reject) => {
this.server = expressApp.listen(port, '0.0.0.0', () => {
Logger.debug(`✌️ HTTP service started successfully`);
metricsService.record('http_service_start', 1, {
port: port.toString(),
});
resolve(this.server);
});
this.server?.on('error', (err: Error) => {
Logger.error('Failed to start HTTP service:', err);
reject(err);
});
});
} catch (err) {
Logger.error('Failed to start HTTP service:', err);
throw err;
}
}
async shutdown() {
try {
if (this.server) {
await new Promise((resolve) => {
this.server?.close(() => {
Logger.debug('HTTP service stopped');
metricsService.record('http_service_stop', 1);
resolve(null);
});
});
}
} catch (err) {
Logger.error('Error while shutting down HTTP service:', err);
throw err;
}
}
getServer() {
return this.server;
}
}
+14
View File
@@ -0,0 +1,14 @@
import path from 'path';
import { Inject, Service } from 'typedi';
import winston from 'winston';
import config from '../config';
@Service()
export default class LogService {
constructor(@Inject('logger') private logger: winston.Logger) {}
public checkFilePath(filePath: string, fileName: string) {
const finalPath = path.resolve(config.logPath, filePath, fileName);
return finalPath.startsWith(config.logPath) ? finalPath : '';
}
}
+92
View File
@@ -0,0 +1,92 @@
import { performance } from 'perf_hooks';
import Logger from '../loaders/logger';
interface Metric {
name: string;
value: number;
timestamp: number;
tags?: Record<string, string>;
}
class MetricsService {
private metrics: Metric[] = [];
private static instance: MetricsService;
private constructor() {
// 定期清理旧数据
setInterval(() => {
const oneHourAgo = Date.now() - 3600000;
this.metrics = this.metrics.filter(m => m.timestamp > oneHourAgo);
}, 60000);
}
static getInstance(): MetricsService {
if (!MetricsService.instance) {
MetricsService.instance = new MetricsService();
}
return MetricsService.instance;
}
record(name: string, value: number, tags?: Record<string, string>) {
this.metrics.push({
name,
value,
timestamp: Date.now(),
tags,
});
}
measure(name: string, fn: () => void, tags?: Record<string, string>) {
const start = performance.now();
try {
fn();
} finally {
const duration = performance.now() - start;
this.record(name, duration, tags);
}
}
async measureAsync(name: string, fn: () => Promise<void>, tags?: Record<string, string>) {
const start = performance.now();
try {
await fn();
} finally {
const duration = performance.now() - start;
this.record(name, duration, tags);
}
}
getMetrics(name?: string, tags?: Record<string, string>) {
let filtered = this.metrics;
if (name) {
filtered = filtered.filter(m => m.name === name);
}
if (tags) {
filtered = filtered.filter(m => {
if (!m.tags) return false;
return Object.entries(tags).every(([key, value]) => m.tags![key] === value);
});
}
return {
count: filtered.length,
average: filtered.reduce((acc, curr) => acc + curr.value, 0) / filtered.length,
min: Math.min(...filtered.map(m => m.value)),
max: Math.max(...filtered.map(m => m.value)),
metrics: filtered,
};
}
report() {
const report = {
timestamp: Date.now(),
metrics: this.getMetrics(),
};
Logger.info('性能指标报告:', report);
return report;
}
}
export const metricsService = MetricsService.getInstance();
+186 -188
View File
@@ -1,11 +1,11 @@
import crypto from 'crypto';
import got from 'got';
import { HttpProxyAgent, HttpsProxyAgent } from 'hpagent';
import nodemailer from 'nodemailer';
import { Inject, Service } from 'typedi';
import { parseBody, parseHeaders } from '../config/util';
import { NotificationInfo } from '../data/notify';
import UserService from './user';
import { httpClient } from '../config/http';
import { ProxyAgent } from 'undici';
@Service()
export default class NotificationService {
@@ -49,17 +49,26 @@ export default class NotificationService {
public async notify(
title: string,
content: string,
notificationInfo?: NotificationInfo,
): Promise<boolean | undefined> {
const { type, ...rest } = await this.userService.getNotificationMode();
let { type, ...rest } = await this.userService.getNotificationMode();
if (notificationInfo?.type) {
type = notificationInfo?.type;
}
if (type) {
this.title = title;
this.content = content;
this.params = rest;
let params = rest;
if (notificationInfo) {
const { type: _, ...others } = notificationInfo;
params = { ...rest, ...others };
}
this.params = params;
const notificationModeAction = this.modeMap.get(type);
try {
return await notificationModeAction?.call(this);
} catch (error: any) {
throw error;
console.error(error);
}
}
return false;
@@ -84,8 +93,9 @@ export default class NotificationService {
private async gotify() {
const { gotifyUrl, gotifyToken, gotifyPriority = 1 } = this.params;
try {
const res: any = await got
.post(`${gotifyUrl}/message?token=${gotifyToken}`, {
const res = await httpClient.post(
`${gotifyUrl}/message?token=${gotifyToken}`,
{
...this.gotOption,
body: `title=${encodeURIComponent(
this.title,
@@ -95,8 +105,8 @@ export default class NotificationService {
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
})
.json();
},
);
if (typeof res.id === 'number') {
return true;
} else {
@@ -110,13 +120,11 @@ export default class NotificationService {
private async goCqHttpBot() {
const { goCqHttpBotQq, goCqHttpBotToken, goCqHttpBotUrl } = this.params;
try {
const res: any = await got
.post(`${goCqHttpBotUrl}?${goCqHttpBotQq}`, {
...this.gotOption,
json: { message: `${this.title}\n${this.content}` },
headers: { Authorization: 'Bearer ' + goCqHttpBotToken },
})
.json();
const res = await httpClient.post(`${goCqHttpBotUrl}?${goCqHttpBotQq}`, {
...this.gotOption,
json: { message: `${this.title}\n${this.content}` },
headers: { Authorization: 'Bearer ' + goCqHttpBotToken },
});
if (res.retcode === 0) {
return true;
} else {
@@ -136,15 +144,13 @@ export default class NotificationService {
: `https://sctapi.ftqq.com/${serverChanKey}.send`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `title=${encodeURIComponent(
this.title,
)}&desp=${encodeURIComponent(this.content)}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `title=${encodeURIComponent(
this.title,
)}&desp=${encodeURIComponent(this.content)}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (res.errno === 0 || res.data.errno === 0) {
return true;
} else {
@@ -159,15 +165,13 @@ export default class NotificationService {
const { pushDeerKey, pushDeerUrl } = this.params;
const url = pushDeerUrl || `https://api2.pushdeer.com/message/push`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `pushkey=${pushDeerKey}&text=${encodeURIComponent(
this.title,
)}&desp=${encodeURIComponent(this.content)}&type=markdown`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `pushkey=${pushDeerKey}&text=${encodeURIComponent(
this.title,
)}&desp=${encodeURIComponent(this.content)}&type=markdown`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (
res.content.result.length !== undefined &&
res.content.result.length > 0
@@ -182,16 +186,13 @@ export default class NotificationService {
}
private async chat() {
const { chatUrl, chatToken } = this.params;
const url = `${chatUrl}${chatToken}`;
const { synologyChatUrl } = this.params;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `payload={"text":"${this.title}\n${this.content}"}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(synologyChatUrl, {
...this.gotOption,
body: `payload={"text":"${this.title}\n${this.content}"}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (res.success) {
return true;
} else {
@@ -227,13 +228,11 @@ export default class NotificationService {
url: barkUrl,
};
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: body,
headers: { 'Content-Type': 'application/json' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
json: body,
headers: { 'Content-Type': 'application/json' },
});
if (res.code === 200) {
return true;
} else {
@@ -259,29 +258,17 @@ export default class NotificationService {
}/bot${telegramBotToken}/sendMessage`;
let agent;
if (telegramBotProxyHost && telegramBotProxyPort) {
const options: any = {
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 256,
maxFreeSockets: 256,
proxy: `http://${authStr}${telegramBotProxyHost}:${telegramBotProxyPort}`,
};
const httpAgent = new HttpProxyAgent(options);
const httpsAgent = new HttpsProxyAgent(options);
agent = {
http: httpAgent,
https: httpsAgent,
};
agent = new ProxyAgent({
uri: `http://${authStr}${telegramBotProxyHost}:${telegramBotProxyPort}`,
});
}
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `chat_id=${telegramBotUserId}&text=${this.title}\n\n${this.content}&disable_web_page_preview=true`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
agent,
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `chat_id=${telegramBotUserId}&text=${this.title}\n\n${this.content}&disable_web_page_preview=true`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
dispatcher: agent,
});
if (res.ok) {
return true;
} else {
@@ -304,17 +291,15 @@ export default class NotificationService {
}
const url = `https://oapi.dingtalk.com/robot/send?access_token=${dingtalkBotToken}${secretParam}`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
const res = await httpClient.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
})
.json();
},
});
if (res.errcode === 0) {
return true;
} else {
@@ -330,17 +315,15 @@ export default class NotificationService {
this.params;
const url = `${weWorkOrigin}/cgi-bin/webhook/send?key=${weWorkBotKey}`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
const res = await httpClient.post(url, {
...this.gotOption,
json: {
msgtype: 'text',
text: {
content: ` ${this.title}\n\n${this.content}`,
},
})
.json();
},
});
if (res.errcode === 0) {
return true;
} else {
@@ -357,15 +340,13 @@ export default class NotificationService {
const [corpid, corpsecret, touser, agentid, thumb_media_id = '1'] =
weWorkAppKey.split(',');
const url = `${weWorkOrigin}/cgi-bin/gettoken`;
const tokenRes: any = await got
.post(url, {
...this.gotOption,
json: {
corpid,
corpsecret,
},
})
.json();
const tokenRes = await httpClient.post(url, {
...this.gotOption,
json: {
corpid,
corpsecret,
},
});
let options: any = {
msgtype: 'mpnews',
@@ -406,20 +387,18 @@ export default class NotificationService {
}
try {
const res: any = await got
.post(
`${weWorkOrigin}/cgi-bin/message/send?access_token=${tokenRes.access_token}`,
{
...this.gotOption,
json: {
touser,
agentid,
safe: '0',
...options,
},
const res = await httpClient.post(
`${weWorkOrigin}/cgi-bin/message/send?access_token=${tokenRes.access_token}`,
{
...this.gotOption,
json: {
touser,
agentid,
safe: '0',
...options,
},
)
.json();
},
);
if (res.errcode === 0) {
return true;
@@ -461,14 +440,12 @@ export default class NotificationService {
}
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
...json,
},
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
json: {
...json,
},
});
if (res.code === 0) {
return true;
} else {
@@ -483,13 +460,11 @@ export default class NotificationService {
const { iGotPushKey } = this.params;
const url = `https://push.hellyw.com/${iGotPushKey.toLowerCase()}`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
body: `title=${this.title}&content=${this.content}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
body: `title=${this.title}&content=${this.content}`,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});
if (res.ret === 0) {
return true;
@@ -528,7 +503,7 @@ export default class NotificationService {
},
};
const res: any = await got.post(url, body).json();
const res = await httpClient.post(url, body);
if (res.code === 200) {
return true;
@@ -552,19 +527,17 @@ export default class NotificationService {
const url = `https://www.weplusbot.com/send`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
token: `${wePlusBotToken}`,
title: `${this.title}`,
template: `${template}`,
content: `${content}`,
receiver: `${wePlusBotReceiver || ''}`,
version: `${wePlusBotVersion || 'pro'}`,
},
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
json: {
token: `${wePlusBotToken}`,
title: `${this.title}`,
template: `${template}`,
content: `${content}`,
receiver: `${wePlusBotReceiver || ''}`,
version: `${wePlusBotVersion || 'pro'}`,
},
});
if (res.code === 200) {
return true;
@@ -584,16 +557,14 @@ export default class NotificationService {
}
try {
const res: any = await got
.post(larkKey, {
...this.gotOption,
json: {
msg_type: 'text',
content: { text: `${this.title}\n\n${this.content}` },
},
headers: { 'Content-Type': 'application/json' },
})
.json();
const res = await httpClient.post(larkKey, {
...this.gotOption,
json: {
msg_type: 'text',
content: { text: `${this.title}\n\n${this.content}` },
},
headers: { 'Content-Type': 'application/json' },
});
if (res.StatusCode === 0 || res.code === 0) {
return true;
} else {
@@ -605,7 +576,7 @@ export default class NotificationService {
}
private async email() {
const { emailPass, emailService, emailUser } = this.params;
const { emailPass, emailService, emailUser, emailTo } = this.params;
try {
const transporter = nodemailer.createTransport({
@@ -618,7 +589,7 @@ export default class NotificationService {
const info = await transporter.sendMail({
from: `"青龙快讯" <${emailUser}>`,
to: `${emailUser}`,
to: emailTo ? emailTo.split(';') : emailUser,
subject: `${this.title}`,
html: `${this.content.replace(/\n/g, '<br/>')}`,
});
@@ -638,19 +609,22 @@ export default class NotificationService {
private async pushMe() {
const { pushMeKey, pushMeUrl } = this.params;
try {
const res: any = await got.post(pushMeUrl || 'https://push.i-i.me/', {
...this.gotOption,
json: {
push_key: pushMeKey,
title: this.title,
content: this.content,
const res = await httpClient.post<'text'>(
pushMeUrl || 'https://push.i-i.me/',
{
...this.gotOption,
json: {
push_key: pushMeKey,
title: this.title,
content: this.content,
},
headers: { 'Content-Type': 'application/json' },
},
headers: { 'Content-Type': 'application/json' },
});
if (res.body === 'success') {
);
if (res === 'success') {
return true;
} else {
throw new Error(res.body);
throw new Error(res);
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
@@ -658,26 +632,49 @@ export default class NotificationService {
}
private async ntfy() {
const { ntfyUrl, ntfyTopic, ntfyPriority } = this.params;
const {
ntfyUrl,
ntfyTopic,
ntfyPriority,
ntfyToken,
ntfyUsername,
ntfyPassword,
ntfyActions,
} = this.params;
// 编码函数
const encodeRfc2047 = (text: string, charset: string = 'UTF-8'): string => {
const encodedText = Buffer.from(text).toString('base64');
return `=?${charset}?B?${encodedText}?=`;
};
try {
const encodedTitle = encodeRfc2047(this.title);
const res: any = await got.post(
const headers: Record<string, string> = {
Title: encodeRfc2047(this.title),
Priority: `${ntfyPriority || '3'}`,
Icon: 'https://qn.whyour.cn/logo.png',
};
if (ntfyToken) {
headers['Authorization'] = `Bearer ${ntfyToken}`;
} else if (ntfyUsername && ntfyPassword) {
headers['Authorization'] = `Basic ${Buffer.from(
`${ntfyUsername}:${ntfyPassword}`,
).toString('base64')}`;
}
if (ntfyActions) {
headers['Actions'] = encodeRfc2047(ntfyActions);
}
const res = await httpClient.request(
`${ntfyUrl || 'https://ntfy.sh'}/${ntfyTopic}`,
{
...this.gotOption,
body: `${this.content}`,
headers: { Title: encodedTitle, Priority: `${ntfyPriority || '3'}` },
headers: headers,
method: 'POST',
},
);
if (res.statusCode === 200) {
return true;
} else {
throw new Error(JSON.stringify(res));
throw new Error(await res.body.text());
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
@@ -711,20 +708,18 @@ export default class NotificationService {
const url = `https://wxpusher.zjiecode.com/api/send/message`;
try {
const res: any = await got
.post(url, {
...this.gotOption,
json: {
appToken: wxPusherBotAppToken,
content: `<h1>${this.title}</h1><br/><div style='white-space: pre-wrap;'>${this.content}</div>`,
summary: this.title,
contentType: 2,
topicIds: topicIds,
uids: uids,
verifyPayType: 0,
},
})
.json();
const res = await httpClient.post(url, {
...this.gotOption,
json: {
appToken: wxPusherBotAppToken,
content: `<h1>${this.title}</h1><br/><div style='white-space: pre-wrap;'>${this.content}</div>`,
summary: this.title,
contentType: 2,
topicIds: topicIds,
uids: uids,
verifyPayType: 0,
},
});
if (res.code === 1000) {
return true;
@@ -775,15 +770,16 @@ export default class NotificationService {
},
],
};
const res: any = await got.post(url, {
const res = await httpClient.request(url, {
...this.gotOption,
json: data,
headers,
method: 'POST',
});
if (res.statusCode === 200) {
return true;
} else {
throw new Error(res.body);
throw new Error(await res.body.text());
}
}
}
@@ -802,7 +798,7 @@ export default class NotificationService {
webhookContentType,
} = this.params;
if (!webhookUrl.includes('$title') && !webhookBody.includes('$title')) {
if (!webhookUrl?.includes('$title') && !webhookBody?.includes('$title')) {
throw new Error('Url 或者 Body 中必须包含 $title');
}
@@ -818,15 +814,17 @@ export default class NotificationService {
allowGetBody: true,
...bodyParam,
};
try {
const formatUrl = webhookUrl
?.replaceAll('$title', encodeURIComponent(this.title))
?.replaceAll('$content', encodeURIComponent(this.content));
const res = await got(formatUrl, options);
const res = await httpClient.request(formatUrl, options);
const text = await res.body.text();
if (String(res.statusCode).startsWith('20')) {
return true;
} else {
throw new Error(JSON.stringify(res));
throw new Error(await res.body.text());
}
} catch (error: any) {
throw new Error(error.response ? error.response.body : error);
+4 -4
View File
@@ -139,7 +139,7 @@ export default class ScheduleService {
) {
const _id = this.formatId(id);
this.logger.info(
'[panel][创建cron任务], 任务ID: %s, cron: %s, 任务名: %s, 执行命令: %s',
'[panel][创建cron任务] 任务ID: %s, cron: %s, 任务名: %s, 执行命令: %s',
_id,
schedule,
name,
@@ -172,7 +172,7 @@ export default class ScheduleService {
async cancelCronTask({ id = 0, name }: ScheduleTaskType) {
const _id = this.formatId(id);
this.logger.info('[panel][取消定时任务], 任务名: %s', name);
this.logger.info('[panel][取消定时任务] 任务名: %s', name);
if (this.scheduleStacks.has(_id)) {
this.scheduleStacks.get(_id)?.cancel();
this.scheduleStacks.delete(_id);
@@ -187,7 +187,7 @@ export default class ScheduleService {
) {
const _id = this.formatId(id);
this.logger.info(
'[panel][创建interval任务], 任务ID: %s, 任务名: %s, 执行命令: %s',
'[panel][创建interval任务] 任务ID: %s, 任务名: %s, 执行命令: %s',
_id,
name,
command,
@@ -232,7 +232,7 @@ export default class ScheduleService {
async cancelIntervalTask({ id = 0, name }: ScheduleTaskType) {
const _id = this.formatId(id);
this.logger.info(
'[panel][取消interval任务], 任务ID: %s, 任务名: %s',
'[panel][取消interval任务] 任务ID: %s, 任务名: %s',
_id,
name,
);
+7 -2
View File
@@ -64,10 +64,15 @@ export default class ScriptService {
return { code: 200 };
}
public async getFile(filePath: string, fileName: string) {
public checkFilePath(filePath: string, fileName: string) {
const finalPath = path.resolve(config.scriptPath, filePath, fileName);
return finalPath.startsWith(config.scriptPath) ? finalPath : '';
}
if (!finalPath.startsWith(config.scriptPath)) {
public async getFile(filePath: string, fileName: string) {
const finalPath = this.checkFilePath(filePath, fileName);
if (!finalPath) {
return '';
}
+6 -2
View File
@@ -26,12 +26,13 @@ export default class SshKeyService {
if (_exist) {
config = await fs.readFile(this.sshConfigFilePath, { encoding: 'utf-8' });
} else {
await writeFileWithLock(this.sshConfigFilePath, '');
await writeFileWithLock(this.sshConfigFilePath, '', { mode: '600' });
}
if (!config.includes(this.sshConfigHeader)) {
await writeFileWithLock(
this.sshConfigFilePath,
`${this.sshConfigHeader}\n\n${config}`,
{ mode: '600' },
);
}
}
@@ -45,7 +46,6 @@ export default class SshKeyService {
path.join(this.sshPath, alias),
`${key}${os.EOL}`,
{
encoding: 'utf8',
mode: '400',
},
);
@@ -81,6 +81,10 @@ export default class SshKeyService {
await writeFileWithLock(
`${path.join(this.sshPath, `${alias}.config`)}`,
config,
{
encoding: 'utf8',
mode: '600',
},
);
}
+39 -8
View File
@@ -42,11 +42,37 @@ export default class SubscriptionService {
private crontabService: CrontabService,
) {}
private addUserIdFilter(query: any, userId?: number) {
if (userId !== undefined) {
query.userId = userId;
}
return query;
}
private async checkOwnership(ids: number[], userId?: number): Promise<void> {
if (userId === undefined) {
// Admin can access all subscriptions
return;
}
const subscriptions = await SubscriptionModel.findAll({
where: { id: ids },
attributes: ['id', 'userId'],
});
const unauthorized = subscriptions.filter(
(sub) => sub.userId !== undefined && sub.userId !== userId
);
if (unauthorized.length > 0) {
throw new Error('无权限操作该订阅');
}
}
public async list(
searchText?: string,
ids?: string,
userId?: number,
): Promise<SubscriptionInstance[]> {
let query = {};
let query: any = {};
this.addUserIdFilter(query, userId);
const subIds = JSON.parse(ids || '[]');
if (searchText) {
const reg = {
@@ -262,7 +288,8 @@ export default class SubscriptionService {
);
}
public async remove(ids: number[], query: { force?: boolean }) {
public async remove(ids: number[], query: { force?: boolean }, userId?: number) {
await this.checkOwnership(ids, userId);
const docs = await SubscriptionModel.findAll({ where: { id: ids } });
for (const doc of docs) {
await this.handleTask(doc.get({ plain: true }), false);
@@ -273,7 +300,7 @@ export default class SubscriptionService {
if (query?.force === true) {
const crons = await CrontabModel.findAll({ where: { sub_id: ids } });
if (crons?.length) {
await this.crontabService.remove(crons.map((x) => x.id!));
await this.crontabService.remove(crons.map((x) => x.id!), userId);
}
for (const doc of docs) {
const filePath = join(config.scriptPath, doc.alias);
@@ -294,7 +321,8 @@ export default class SubscriptionService {
return doc.get({ plain: true });
}
public async run(ids: number[]) {
public async run(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await SubscriptionModel.update(
{ status: SubscriptionStatus.queued },
{ where: { id: ids } },
@@ -304,7 +332,8 @@ export default class SubscriptionService {
});
}
public async stop(ids: number[]) {
public async stop(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
const docs = await SubscriptionModel.findAll({ where: { id: ids } });
for (const doc of docs) {
if (doc.pid) {
@@ -339,7 +368,8 @@ export default class SubscriptionService {
});
}
public async disabled(ids: number[]) {
public async disabled(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await SubscriptionModel.update({ is_disabled: 1 }, { where: { id: ids } });
const docs = await SubscriptionModel.findAll({ where: { id: ids } });
await this.setSshConfig();
@@ -348,7 +378,8 @@ export default class SubscriptionService {
}
}
public async enabled(ids: number[]) {
public async enabled(ids: number[], userId?: number) {
await this.checkOwnership(ids, userId);
await SubscriptionModel.update({ is_disabled: 0 }, { where: { id: ids } });
const docs = await SubscriptionModel.findAll({ where: { id: ids } });
await this.setSshConfig();
@@ -384,7 +415,7 @@ export default class SubscriptionService {
files.map(async (x) => ({
filename: x,
directory: relativeDir.replace(config.logPath, ''),
time: (await fs.lstat(`${dir}/${x}`)).mtime.getTime(),
time: (await fs.lstat(`${dir}/${x}`)).birthtimeMs,
})),
)
).sort((a, b) => b.time - a.time);
+84 -15
View File
@@ -1,13 +1,13 @@
import { spawn } from 'cross-spawn';
import { Response } from 'express';
import fs from 'fs';
import got from 'got';
import { Agent, request } from 'undici';
import sum from 'lodash/sum';
import path from 'path';
import { Inject, Service } from 'typedi';
import winston from 'winston';
import config from '../config';
import { TASK_COMMAND } from '../config/const';
import { NotificationModeStringMap, TASK_COMMAND } from '../config/const';
import {
getPid,
killTask,
@@ -16,6 +16,7 @@ import {
promiseExec,
readDirs,
rmPath,
setSystemTimezone,
} from '../config/util';
import {
DependenceModel,
@@ -46,11 +47,14 @@ export default class SystemService {
@Inject('logger') private logger: winston.Logger,
private scheduleService: ScheduleService,
private sockService: SockService,
) {}
) { }
public async getSystemConfig() {
const doc = await this.getDb({ type: AuthDataType.systemConfig });
return doc;
return {
...doc,
info: { ...doc.info, timezone: doc.info?.timezone || 'Asia/Shanghai' },
};
}
private async updateAuthDb(payload: SystemInfo): Promise<SystemInfo> {
@@ -272,14 +276,18 @@ export default class SystemService {
let lastVersionContent;
try {
const result = await got.get(
const { body } = await request(
`${config.lastVersionFile}?t=${Date.now()}`,
{
timeout: 30000,
dispatcher: new Agent({
keepAliveTimeout: 30000,
keepAliveMaxTimeout: 30000,
}),
},
);
lastVersionContent = parseContentVersion(result.body);
} catch (error) {}
const text = await body.text();
lastVersionContent = parseContentVersion(text);
} catch (error) { }
if (!lastVersionContent) {
lastVersionContent = currentVersionContent;
@@ -353,13 +361,39 @@ export default class SystemService {
public async reloadSystem(target?: 'system' | 'data') {
const cmd = `real_time=true ql reload ${target || ''}`;
const cp = spawn(cmd, { shell: '/bin/bash' });
const cp = spawn(cmd, {
shell: '/bin/bash',
detached: true,
stdio: 'ignore',
});
cp.unref();
setTimeout(() => {
process.exit(0);
});
return { code: 200 };
}
public async notify({ title, content }: { title: string; content: string }) {
const isSuccess = await this.notificationService.notify(title, content);
public async notify({
title,
content,
notificationInfo,
}: {
title: string;
content: string;
notificationInfo?: NotificationInfo;
}) {
const typeString =
typeof notificationInfo?.type === 'number'
? NotificationModeStringMap[notificationInfo.type]
: undefined;
if (notificationInfo && typeString) {
notificationInfo.type = typeString;
}
const isSuccess = await this.notificationService.notify(
title,
content,
notificationInfo,
);
if (isSuccess) {
return { code: 200, message: '通知发送成功' };
} else {
@@ -367,11 +401,12 @@ export default class SystemService {
}
}
public async run({ command }: { command: string }, callback: TaskCallbacks) {
public async run({ command, logPath }: { command: string; logPath?: string }, callback: TaskCallbacks) {
if (!command.startsWith(TASK_COMMAND)) {
command = `${TASK_COMMAND} ${command}`;
}
this.scheduleService.runTask(`real_time=true ${command}`, callback, {
const logPathPrefix = logPath ? `real_log_path=${logPath}` : ''
this.scheduleService.runTask(`${logPathPrefix} real_time=true ${command}`, callback, {
command,
id: command.replace(/ /g, '-'),
runOrigin: 'system',
@@ -400,10 +435,16 @@ export default class SystemService {
}
}
public async exportData(res: Response) {
public async exportData(res: Response, type?: string[]) {
try {
let dataDirs = ['db', 'upload'];
if (type && type.length) {
dataDirs = dataDirs.concat(type.filter((x) => x !== 'base'));
}
const dataPaths = dataDirs.map((dir) => `data/${dir}`);
await promiseExec(
`cd ${config.dataPath} && cd ../ && tar -zcvf ${config.dataTgzFile} data/`,
`cd ${config.dataPath} && cd ../ && tar -zcvf ${config.dataTgzFile
} ${dataPaths.join(' ')}`,
);
res.download(config.dataTgzFile);
} catch (error: any) {
@@ -471,4 +512,32 @@ export default class SystemService {
await rmPath(path.join(config.systemLogPath, log.title));
}
}
public async updateTimezone(info: SystemModelInfo) {
if (!info.timezone) {
info.timezone = 'Asia/Shanghai';
}
const oDoc = await this.getSystemConfig();
await this.updateAuthDb({
...oDoc,
info: { ...oDoc.info, ...info },
});
const success = await setSystemTimezone(info.timezone);
if (success) {
return { code: 200, data: info };
} else {
return { code: 400, message: '设置时区失败' };
}
}
public async cleanDependence(type: 'node' | 'python3') {
if (!type || !['node', 'python3'].includes(type)) {
return { code: 400, message: '参数错误' };
}
try {
const finalPath = path.join(config.dependenceCachePath, type);
await fs.promises.rm(finalPath, { recursive: true });
} catch (error) { }
return { code: 200 };
}
}
+85 -25
View File
@@ -1,6 +1,6 @@
import { Service, Inject } from 'typedi';
import winston from 'winston';
import { createRandomString, getNetIp } from '../config/util';
import { createRandomString } from '../config/util';
import config from '../config';
import jwt from 'jsonwebtoken';
import { authenticator } from '@otplib/preset-default';
@@ -21,13 +21,20 @@ import dayjs from 'dayjs';
import IP2Region from 'ip2region';
import requestIp from 'request-ip';
import uniq from 'lodash/uniq';
import pickBy from 'lodash/pickBy';
import isNil from 'lodash/isNil';
import { shareStore } from '../shared/store';
import UserManagementService from './userManagement';
import { UserRole } from '../data/user';
@Service()
export default class UserService {
@Inject((type) => NotificationService)
private notificationService!: NotificationService;
@Inject((type) => UserManagementService)
private userManagementService!: UserManagementService;
constructor(
@Inject('logger') private logger: winston.Logger,
private scheduleService: ScheduleService,
@@ -91,27 +98,57 @@ export default class UserService {
const { country, province, city, isp } = ipAddress;
address = uniq([country, province, city, isp]).filter(Boolean).join(' ');
}
if (username === cUsername && password === cPassword) {
// Check if this is a regular user (not admin) trying to login
let authenticatedUser = null;
let userId: number | undefined = undefined;
let userRole = UserRole.admin;
// First check if it's the system admin
const isSystemAdmin = username === cUsername && password === cPassword;
if (!isSystemAdmin) {
// Try to authenticate as a regular user
try {
authenticatedUser = await this.userManagementService.authenticate(username, password);
if (authenticatedUser) {
userId = authenticatedUser.id;
userRole = authenticatedUser.role;
}
} catch (e: any) {
// User disabled or other error
return { code: 400, message: e.message };
}
}
if (isSystemAdmin || authenticatedUser) {
const data = createRandomString(50, 100);
const expiration = twoFactorActivated ? 60 : 20;
let token = jwt.sign({ data }, config.secret as any, {
expiresIn: 60 * 60 * 24 * expiration,
const expiration = (isSystemAdmin && twoFactorActivated) ? '60d' : '20d';
let token = jwt.sign(
{ data, userId, role: userRole },
config.jwt.secret,
{
expiresIn: config.jwt.expiresIn || expiration,
algorithm: 'HS384',
});
await this.updateAuthInfo(content, {
token,
tokens: {
...tokens,
[req.platform]: token,
},
lastlogon: timestamp,
retries: 0,
lastip: ip,
lastaddr: address,
platform: req.platform,
isTwoFactorChecking: false,
});
// Only update authInfo for system admin
if (isSystemAdmin) {
await this.updateAuthInfo(content, {
token,
tokens: {
...tokens,
[req.platform]: token,
},
lastlogon: timestamp,
retries: 0,
lastip: ip,
lastaddr: address,
platform: req.platform,
isTwoFactorChecking: false,
});
}
this.notificationService.notify(
'登录通知',
`你于${dayjs(timestamp).format('YYYY-MM-DD HH:mm:ss')}${address} ${
@@ -131,7 +168,14 @@ export default class UserService {
this.getLoginLog();
return {
code: 200,
data: { token, lastip, lastaddr, lastlogon, retries, platform },
data: {
token,
lastip,
lastaddr,
lastlogon,
retries,
platform,
},
};
} else {
await this.updateAuthInfo(content, {
@@ -264,7 +308,16 @@ export default class UserService {
if (isValid) {
return this.login({ username, password }, req, false);
} else {
const { ip, address } = await getNetIp(req);
const ip = requestIp.getClientIp(req) || '';
const query = new IP2Region();
const ipAddress = query.search(ip);
let address = '';
if (ipAddress) {
const { country, province, city, isp } = ipAddress;
address = uniq([country, province, city, isp])
.filter(Boolean)
.join(' ');
}
await this.updateAuthInfo(authInfo, {
lastip: ip,
lastaddr: address,
@@ -347,11 +400,18 @@ export default class UserService {
}
public async resetAuthInfo(info: Partial<AuthInfo>) {
const { retries, twoFactorActivated } = info;
const { retries, twoFactorActivated, password, username } = info;
const authInfo = await this.getAuthInfo();
await this.updateAuthInfo(authInfo, {
retries,
twoFactorActivated,
});
const payload = pickBy(
{
retries,
twoFactorActivated,
password,
username,
},
(x) => !isNil(x),
);
await this.updateAuthInfo(authInfo, payload);
}
}
+125
View File
@@ -0,0 +1,125 @@
import { Service, Inject } from 'typedi';
import winston from 'winston';
import { User, UserModel, UserRole, UserStatus } from '../data/user';
import { Op } from 'sequelize';
import bcrypt from 'bcrypt';
@Service()
export default class UserManagementService {
constructor(@Inject('logger') private logger: winston.Logger) {}
private async hashPassword(password: string): Promise<string> {
const saltRounds = 10;
return bcrypt.hash(password, saltRounds);
}
private async verifyPassword(password: string, hash: string): Promise<boolean> {
return bcrypt.compare(password, hash);
}
public async list(searchText?: string): Promise<User[]> {
let query: any = {};
if (searchText) {
query = {
username: { [Op.like]: `%${searchText}%` },
};
}
const docs = await UserModel.findAll({ where: query });
return docs.map((x) => x.get({ plain: true }));
}
public async get(id: number): Promise<User> {
const doc = await UserModel.findByPk(id);
if (!doc) {
throw new Error('用户不存在');
}
return doc.get({ plain: true });
}
public async getByUsername(username: string): Promise<User | null> {
const doc = await UserModel.findOne({ where: { username } });
if (!doc) {
return null;
}
return doc.get({ plain: true });
}
public async create(payload: User): Promise<User> {
const existingUser = await this.getByUsername(payload.username);
if (existingUser) {
throw new Error('用户名已存在');
}
if (payload.password.length < 6) {
throw new Error('密码长度至少为6位');
}
// Hash the password before storing
const hashedPassword = await this.hashPassword(payload.password);
const userWithHashedPassword = { ...payload, password: hashedPassword };
const doc = await UserModel.create(userWithHashedPassword);
return doc.get({ plain: true });
}
public async update(payload: User): Promise<User> {
if (!payload.id) {
throw new Error('缺少用户ID');
}
const existingUser = await this.get(payload.id);
if (!existingUser) {
throw new Error('用户不存在');
}
if (payload.password && payload.password.length < 6) {
throw new Error('密码长度至少为6位');
}
// Check if username is being changed and if new username already exists
if (payload.username !== existingUser.username) {
const userWithSameUsername = await this.getByUsername(payload.username);
if (userWithSameUsername && userWithSameUsername.id !== payload.id) {
throw new Error('用户名已存在');
}
}
// Hash the password if it's being updated
const updatePayload = { ...payload };
if (payload.password) {
updatePayload.password = await this.hashPassword(payload.password);
}
const [, [updated]] = await UserModel.update(updatePayload, {
where: { id: payload.id },
returning: true,
});
return updated.get({ plain: true });
}
public async delete(ids: number[]): Promise<number> {
const count = await UserModel.destroy({ where: { id: ids } });
return count;
}
public async authenticate(
username: string,
password: string,
): Promise<User | null> {
const user = await this.getByUsername(username);
if (!user) {
return null;
}
const isPasswordValid = await this.verifyPassword(password, user.password);
if (!isPasswordValid) {
return null;
}
if (user.status === UserStatus.disabled) {
throw new Error('用户已被禁用');
}
return user;
}
}
+14 -1
View File
@@ -37,7 +37,7 @@ class TaskLimit {
concurrency: Math.max(os.cpus().length, 4),
});
private client = new ApiClient(
`0.0.0.0:${config.cronPort}`,
`0.0.0.0:${config.grpcPort}`,
credentials.createInsecure(),
{ 'grpc.enable_http_proxy': 0 },
);
@@ -189,6 +189,19 @@ class TaskLimit {
return this.scriptLimit.add(fn, options);
}
public async waitDependencyQueueDone(): Promise<void> {
if (this.dependenyLimit.size === 0 && this.dependenyLimit.pending === 0) {
return;
}
return new Promise((resolve) => {
const onIdle = () => {
this.dependenyLimit.removeListener('idle', onIdle);
resolve();
};
this.dependenyLimit.on('idle', onIdle);
});
}
public runDependeny<T>(
dependency: Dependence,
fn: IDependencyFn<T>,
+1 -1
View File
@@ -13,7 +13,7 @@ function getUniqueLockPath(filePath: string) {
export async function writeFileWithLock(
filePath: string,
content: string | Buffer,
content: string,
options: Parameters<typeof writeFile>[2] = {},
) {
if (typeof options === 'string') {
-1
View File
@@ -2,7 +2,6 @@ import 'reflect-metadata';
import OpenService from './services/open';
import { Container } from 'typedi';
import LoggerInstance from './loaders/logger';
import fs from 'fs';
import config from './config';
import path from 'path';
import os from 'os';
+8 -5
View File
@@ -2,7 +2,11 @@
"compilerOptions": {
"target": "es2017",
"lib": ["ESNext"],
"typeRoots": ["./node_modules/celebrate/lib", "./node_modules/@types"],
"typeRoots": [
"./types",
"../node_modules/celebrate/lib",
"../node_modules/@types"
],
"allowSyntheticDefaultImports": true,
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
@@ -13,12 +17,11 @@
"module": "commonjs",
"pretty": true,
"sourceMap": true,
"outDir": "./static/build",
"outDir": "../static/build",
"allowJs": true,
"noEmit": false,
"esModuleInterop": true
},
"include": ["./back/**/*", "./back.d.ts"],
"exclude": ["node_modules"],
"files": ["./back/index.d.ts"]
"include": ["./**/*"],
"exclude": ["node_modules"]
}
+16
View File
@@ -0,0 +1,16 @@
/// <reference types="express" />
export {};
declare global {
namespace Express {
interface Request {
platform: 'desktop' | 'mobile';
user?: {
userId?: number;
role?: number;
};
auth?: any;
}
}
}
-27
View File
@@ -1,27 +0,0 @@
import 'reflect-metadata'; // We need this in order to use @Decorators
import config from './config';
import express from 'express';
import depInjectorLoader from './loaders/depInjector';
import Logger from './loaders/logger';
async function startServer() {
const app = express();
depInjectorLoader();
await require('./loaders/update').default({ app });
app
.listen(config.updatePort, '0.0.0.0', () => {
Logger.debug(`✌️ 更新服务启动成功!`);
console.debug(`✌️ 更新服务启动成功!`);
process.send?.('ready');
})
.on('error', (err) => {
Logger.error(err);
console.error(err);
process.exit(1);
});
}
startServer();
+80
View File
@@ -0,0 +1,80 @@
import { Joi } from 'celebrate';
import cron_parser from 'cron-parser';
import { ScheduleType } from '../interface/schedule';
import path from 'path';
import config from '../config';
const validateSchedule = (value: string, helpers: any) => {
if (
value.startsWith(ScheduleType.ONCE) ||
value.startsWith(ScheduleType.BOOT)
) {
return value;
}
try {
if (cron_parser.parseExpression(value).hasNext()) {
return value;
}
} catch (e) {
return helpers.error('any.invalid');
}
return helpers.error('any.invalid');
};
export const scheduleSchema = Joi.string()
.required()
.custom(validateSchedule)
.messages({
'any.invalid': '无效的定时规则',
'string.empty': '定时规则不能为空',
});
export const commonCronSchema = {
name: Joi.string().optional(),
command: Joi.string().required(),
schedule: scheduleSchema,
labels: Joi.array().optional(),
sub_id: Joi.number().optional().allow(null),
extra_schedules: Joi.array().optional().allow(null),
task_before: Joi.string().optional().allow('').allow(null),
task_after: Joi.string().optional().allow('').allow(null),
log_name: Joi.string()
.optional()
.allow('')
.allow(null)
.custom((value, helpers) => {
if (!value) return value;
// Check if it's an absolute path
if (value.startsWith('/')) {
// Allow /dev/null as special case
if (value === '/dev/null') {
return value;
}
// For other absolute paths, ensure they are within the safe log directory
const normalizedValue = path.normalize(value);
const normalizedLogPath = path.normalize(config.logPath);
if (!normalizedValue.startsWith(normalizedLogPath)) {
return helpers.error('string.unsafePath');
}
return value;
}
if (!/^(?!.*(?:^|\/)\.{1,2}(?:\/|$))(?:\/)?(?:[\w.-]+\/)*[\w.-]+\/?$/.test(value)) {
return helpers.error('string.pattern.base');
}
if (value.length > 100) {
return helpers.error('string.max');
}
return value;
})
.messages({
'string.pattern.base': '日志名称只能包含字母、数字、下划线和连字符',
'string.max': '日志名称不能超过100个字符',
'string.unsafePath': '绝对路径必须在日志目录内或使用 /dev/null',
}),
};
+18 -13
View File
@@ -13,15 +13,13 @@ ARG QL_MAINTAINER="whyour"
LABEL maintainer="${QL_MAINTAINER}"
ARG QL_URL=https://github.com/${QL_MAINTAINER}/qinglong.git
ARG QL_BRANCH=develop
ARG PYTHON_SHORT_VERSION=3.10
ENV PNPM_HOME=/root/.local/share/pnpm \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/share/pnpm:/root/.local/share/pnpm/global/5/node_modules \
NODE_PATH=/usr/local/bin:/usr/local/pnpm-global/5/node_modules:/usr/local/lib/node_modules:/root/.local/share/pnpm/global/5/node_modules \
ENV QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH} \
LANG=C.UTF-8 \
SHELL=/bin/bash \
PS1="\u@\h:\w \$ " \
QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH}
PS1="\u@\h:\w \$ "
VOLUME /ql/data
@@ -41,7 +39,6 @@ RUN set -x \
tzdata \
perl \
openssl \
nginx \
nodejs \
jq \
openssh \
@@ -53,14 +50,11 @@ RUN set -x \
&& apk update \
&& ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezone \
&& git config --global user.email "qinglong@@users.noreply.github.com" \
&& git config --global user.email "qinglong@users.noreply.github.com" \
&& git config --global user.name "qinglong" \
&& git config --global http.postBuffer 524288000 \
&& rm -rf /root/.pnpm-store \
&& rm -rf /root/.local/share/pnpm/store \
&& rm -rf /root/.cache \
&& ulimit -c 0 \
&& pip3 install requests
&& ulimit -c 0
ARG SOURCE_COMMIT
RUN git clone --depth=1 -b ${QL_BRANCH} ${QL_URL} ${QL_DIR} \
@@ -73,11 +67,22 @@ RUN git clone --depth=1 -b ${QL_BRANCH} ${QL_URL} ${QL_DIR} \
&& cp -rf /static/* ${QL_DIR}/static \
&& rm -rf /static
ENV PNPM_HOME=${QL_DIR}/data/dep_cache/node \
PYTHON_HOME=${QL_DIR}/data/dep_cache/python3 \
PYTHONUSERBASE=${QL_DIR}/data/dep_cache/python3
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PNPM_HOME}:${PYTHON_HOME}/bin \
NODE_PATH=/usr/local/bin:/usr/local/lib/node_modules:${PNPM_HOME}/global/5/node_modules \
PIP_CACHE_DIR=${PYTHON_HOME}/pip \
PYTHONPATH=${PYTHON_HOME}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}/site-packages
RUN pip3 install --prefix ${PYTHON_HOME} requests
COPY --from=builder /tmp/build/node_modules/. /ql/node_modules/
WORKDIR ${QL_DIR}
HEALTHCHECK --interval=5s --timeout=2s --retries=20 \
CMD curl -sf --noproxy '*' http://127.0.0.1:5400/api/health || exit 1
CMD curl -sf --noproxy '*' http://127.0.0.1:5700/api/health || exit 1
ENTRYPOINT ["./docker/docker-entrypoint.sh"]
+18 -13
View File
@@ -13,15 +13,13 @@ ARG QL_MAINTAINER="whyour"
LABEL maintainer="${QL_MAINTAINER}"
ARG QL_URL=https://github.com/${QL_MAINTAINER}/qinglong.git
ARG QL_BRANCH=develop
ARG PYTHON_SHORT_VERSION=3.11
ENV PNPM_HOME=/root/.local/share/pnpm \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/share/pnpm:/root/.local/share/pnpm/global/5/node_modules \
NODE_PATH=/usr/local/bin:/usr/local/pnpm-global/5/node_modules:/usr/local/lib/node_modules:/root/.local/share/pnpm/global/5/node_modules \
ENV QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH} \
LANG=C.UTF-8 \
SHELL=/bin/bash \
PS1="\u@\h:\w \$ " \
QL_DIR=/ql \
QL_BRANCH=${QL_BRANCH}
PS1="\u@\h:\w \$ "
VOLUME /ql/data
@@ -41,7 +39,6 @@ RUN set -x \
tzdata \
perl \
openssl \
nginx \
nodejs \
jq \
openssh \
@@ -53,14 +50,11 @@ RUN set -x \
&& apk update \
&& ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezone \
&& git config --global user.email "qinglong@@users.noreply.github.com" \
&& git config --global user.email "qinglong@users.noreply.github.com" \
&& git config --global user.name "qinglong" \
&& git config --global http.postBuffer 524288000 \
&& rm -rf /root/.pnpm-store \
&& rm -rf /root/.local/share/pnpm/store \
&& rm -rf /root/.cache \
&& ulimit -c 0 \
&& pip3 install requests
&& ulimit -c 0
ARG SOURCE_COMMIT
RUN git clone --depth=1 -b ${QL_BRANCH} ${QL_URL} ${QL_DIR} \
@@ -73,11 +67,22 @@ RUN git clone --depth=1 -b ${QL_BRANCH} ${QL_URL} ${QL_DIR} \
&& cp -rf /static/* ${QL_DIR}/static \
&& rm -rf /static
ENV PNPM_HOME=${QL_DIR}/data/dep_cache/node \
PYTHON_HOME=${QL_DIR}/data/dep_cache/python3 \
PYTHONUSERBASE=${QL_DIR}/data/dep_cache/python3
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PNPM_HOME}:${PYTHON_HOME}/bin \
NODE_PATH=/usr/local/bin:/usr/local/lib/node_modules:${PNPM_HOME}/global/5/node_modules \
PIP_CACHE_DIR=${PYTHON_HOME}/pip \
PYTHONPATH=${PYTHON_HOME}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}:${PYTHON_HOME}/lib/python${PYTHON_SHORT_VERSION}/site-packages
RUN pip3 install --prefix ${PYTHON_HOME} requests
COPY --from=builder /tmp/build/node_modules/. /ql/node_modules/
WORKDIR ${QL_DIR}
HEALTHCHECK --interval=5s --timeout=2s --retries=20 \
CMD curl -sf --noproxy '*' http://127.0.0.1:5400/api/health || exit 1
CMD curl -sf --noproxy '*' http://127.0.0.1:5700/api/health || exit 1
ENTRYPOINT ["./docker/docker-entrypoint.sh"]
+13 -20
View File
@@ -4,41 +4,34 @@ dir_shell=/ql/shell
. $dir_shell/share.sh
. $dir_shell/env.sh
echo -e "======================1. 检测配置文件========================\n"
log_with_style() {
local level="$1"
local message="$2"
local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
printf "\n[%s] [%7s] %s\n" "${timestamp}" "${level}" "${message}"
}
log_with_style "INFO" "🚀 1. 检测配置文件..."
import_config "$@"
make_dir /etc/nginx/conf.d
make_dir /run/nginx
init_nginx
fix_config
pm2 l &>/dev/null
echo -e "======================2. 安装依赖========================\n"
patch_version
echo -e "======================3. 启动nginx========================\n"
nginx -s reload 2>/dev/null || nginx -c /etc/nginx/nginx.conf
echo -e "nginx启动成功...\n"
echo -e "======================4. 启动pm2服务========================\n"
reload_update
log_with_style "INFO" "⚙️ 2. 启动 pm2 服务..."
reload_pm2
if [[ $AutoStartBot == true ]]; then
echo -e "======================5. 启动bot========================\n"
log_with_style "INFO" "🤖 3. 启动 bot..."
nohup ql bot >$dir_log/bot.log 2>&1 &
echo -e "bot后台启动中...\n"
fi
if [[ $EnableExtraShell == true ]]; then
echo -e "====================6. 执行自定义脚本========================\n"
log_with_style "INFO" "🛠️ 4. 执行自定义脚本..."
nohup ql extra >$dir_log/extra.log 2>&1 &
echo -e "自定义脚本后台执行中...\n"
fi
echo -e "############################################################\n"
echo -e "容器启动成功..."
echo -e "############################################################\n"
log_with_style "SUCCESS" "🎉 容器启动成功!"
crond -f >/dev/null
-93
View File
@@ -1,93 +0,0 @@
upstream baseApi {
server 0.0.0.0:5600;
}
upstream publicApi {
server 0.0.0.0:5400;
}
upstream updateApi {
server 0.0.0.0:5300;
}
map $http_upgrade $connection_upgrade {
default keep-alive;
'websocket' upgrade;
}
server {
IPV4_CONFIG
IPV6_CONFIG
ssl_session_timeout 5m;
location QL_BASE_URLapi/update/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://updateApi/api/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
}
location QL_BASE_URLapi/public/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://publicApi/api/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
}
location QL_BASE_URLapi/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://baseApi/api/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location QL_BASE_URLopen/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://baseApi/open/;
proxy_buffering off;
proxy_redirect default;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
proxy_read_timeout 1800;
}
gzip on;
gzip_static on;
gzip_types text/plain application/json application/javascript application/x-javascript text/css application/xml text/javascript;
gzip_proxied any;
gzip_vary on;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.0;
QL_ROOT_CONFIG
location QL_BASE_URL_LOCATION {
QL_ALIAS_CONFIG
index index.html index.htm;
try_files $uri QL_BASE_URLindex.html;
}
location ~ .*\.(html)$ {
add_header Cache-Control no-cache;
}
}
-45
View File
@@ -1,45 +0,0 @@
user root;
worker_processes auto;
pcre_jit on;
error_log /var/log/nginx/error.log warn;
include /etc/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
server_tokens off;
client_max_body_size 4096m;
client_body_buffer_size 20m;
keepalive_timeout 65;
sendfile on;
tcp_nodelay on;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:2m;
gzip on;
gzip_static on;
gzip_types text/plain application/json application/javascript application/x-javascript text/css application/xml text/javascript;
gzip_proxied any;
gzip_vary on;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.0;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
include /etc/nginx/conf.d/*.conf;
}
+5 -25
View File
@@ -1,14 +1,14 @@
module.exports = {
apps: [
{
name: 'schedule',
max_restarts: 10,
kill_timeout: 15000,
name: 'qinglong',
max_restarts: 5,
kill_timeout: 1000,
wait_ready: true,
listen_timeout: 10000,
listen_timeout: 5000,
source_map_support: true,
time: true,
script: 'static/build/schedule/index.js',
script: 'static/build/app.js',
env: {
http_proxy: '',
https_proxy: '',
@@ -18,25 +18,5 @@ module.exports = {
ALL_PROXY: '',
},
},
{
name: 'public',
max_restarts: 10,
kill_timeout: 15000,
wait_ready: true,
listen_timeout: 10000,
source_map_support: true,
time: true,
script: 'static/build/public.js',
},
{
name: 'panel',
max_restarts: 10,
kill_timeout: 15000,
wait_ready: true,
listen_timeout: 10000,
source_map_support: true,
time: true,
script: 'static/build/app.js',
},
],
};
+317
View File
@@ -0,0 +1,317 @@
#!/usr/bin/env node
/**
* Multi-User Data Migration Script
*
* This script migrates existing data (Cron, Env, Subscription, Dependence)
* to be associated with specific users.
*
* Usage:
* node migrate-to-multiuser.js --userId=1
* node migrate-to-multiuser.js --username=admin
* node migrate-to-multiuser.js --list-users
*
* Options:
* --userId=<id> Assign all legacy data to user with this ID
* --username=<name> Assign all legacy data to user with this username
* --list-users List all users in the system
* --dry-run Show what would be changed without making changes
* --help Show this help message
*/
const path = require('path');
const fs = require('fs');
const Sequelize = require('sequelize');
// Load environment variables
require('dotenv').config();
// Configuration
const config = {
dbPath: process.env.QL_DATA_DIR || path.join(__dirname, '../data'),
rootPath: __dirname,
};
// Initialize Sequelize
const sequelize = new Sequelize({
dialect: 'sqlite',
storage: path.join(config.dbPath, 'database.sqlite'),
logging: false,
});
// Define models
const UserModel = sequelize.define('User', {
username: Sequelize.STRING,
password: Sequelize.STRING,
role: Sequelize.NUMBER,
status: Sequelize.NUMBER,
});
const CrontabModel = sequelize.define('Crontab', {
name: Sequelize.STRING,
command: Sequelize.STRING,
schedule: Sequelize.STRING,
userId: Sequelize.NUMBER,
});
const EnvModel = sequelize.define('Env', {
name: Sequelize.STRING,
value: Sequelize.STRING,
userId: Sequelize.NUMBER,
});
const SubscriptionModel = sequelize.define('Subscription', {
name: Sequelize.STRING,
url: Sequelize.STRING,
userId: Sequelize.NUMBER,
});
const DependenceModel = sequelize.define('Dependence', {
name: Sequelize.STRING,
type: Sequelize.NUMBER,
userId: Sequelize.NUMBER,
});
// Parse command line arguments
function parseArgs() {
const args = {
userId: null,
username: null,
listUsers: false,
dryRun: false,
help: false,
};
process.argv.slice(2).forEach(arg => {
if (arg.startsWith('--userId=')) {
args.userId = parseInt(arg.split('=')[1]);
} else if (arg.startsWith('--username=')) {
args.username = arg.split('=')[1];
} else if (arg === '--list-users') {
args.listUsers = true;
} else if (arg === '--dry-run') {
args.dryRun = true;
} else if (arg === '--help' || arg === '-h') {
args.help = true;
}
});
return args;
}
// Show help
function showHelp() {
console.log(`
Multi-User Data Migration Script
This script migrates existing data (Cron, Env, Subscription, Dependence)
to be associated with specific users.
Usage:
node migrate-to-multiuser.js --userId=1
node migrate-to-multiuser.js --username=admin
node migrate-to-multiuser.js --list-users
Options:
--userId=<id> Assign all legacy data to user with this ID
--username=<name> Assign all legacy data to user with this username
--list-users List all users in the system
--dry-run Show what would be changed without making changes
--help Show this help message
Examples:
# List all users
node migrate-to-multiuser.js --list-users
# Migrate all data to user ID 1 (dry run)
node migrate-to-multiuser.js --userId=1 --dry-run
# Migrate all data to user 'admin'
node migrate-to-multiuser.js --username=admin
Note: This script will only migrate data where userId is NULL or undefined.
Data already assigned to users will not be changed.
`);
}
// List all users
async function listUsers() {
const users = await UserModel.findAll();
if (users.length === 0) {
console.log('\nNo users found in the database.');
console.log('Please create users first using the User Management interface.');
return;
}
console.log('\nUsers in the system:');
console.log('ID\tUsername\tRole\t\tStatus');
console.log('--\t--------\t----\t\t------');
users.forEach(user => {
const role = user.role === 0 ? 'Admin' : 'User';
const status = user.status === 0 ? 'Enabled' : 'Disabled';
console.log(`${user.id}\t${user.username}\t\t${role}\t\t${status}`);
});
console.log('');
}
// Get statistics of legacy data
async function getStatistics() {
const stats = {
crons: await CrontabModel.count({ where: { userId: null } }),
envs: await EnvModel.count({ where: { userId: null } }),
subscriptions: await SubscriptionModel.count({ where: { userId: null } }),
dependences: await DependenceModel.count({ where: { userId: null } }),
};
return stats;
}
// Migrate data to a specific user
async function migrateData(userId, dryRun = false) {
const stats = await getStatistics();
console.log('\nLegacy Data Statistics:');
console.log(` Cron tasks: ${stats.crons}`);
console.log(` Environment variables: ${stats.envs}`);
console.log(` Subscriptions: ${stats.subscriptions}`);
console.log(` Dependencies: ${stats.dependences}`);
console.log('');
if (stats.crons + stats.envs + stats.subscriptions + stats.dependences === 0) {
console.log('No legacy data found. All data is already assigned to users.');
return;
}
if (dryRun) {
console.log('DRY RUN: No changes will be made.\n');
console.log(`Would assign all legacy data to user ID ${userId}`);
return;
}
console.log(`Migrating data to user ID ${userId}...`);
const transaction = await sequelize.transaction();
try {
// Migrate crons
if (stats.crons > 0) {
await CrontabModel.update(
{ userId },
{ where: { userId: null }, transaction }
);
console.log(`✓ Migrated ${stats.crons} cron tasks`);
}
// Migrate envs
if (stats.envs > 0) {
await EnvModel.update(
{ userId },
{ where: { userId: null }, transaction }
);
console.log(`✓ Migrated ${stats.envs} environment variables`);
}
// Migrate subscriptions
if (stats.subscriptions > 0) {
await SubscriptionModel.update(
{ userId },
{ where: { userId: null }, transaction }
);
console.log(`✓ Migrated ${stats.subscriptions} subscriptions`);
}
// Migrate dependences
if (stats.dependences > 0) {
await DependenceModel.update(
{ userId },
{ where: { userId: null }, transaction }
);
console.log(`✓ Migrated ${stats.dependences} dependencies`);
}
await transaction.commit();
console.log('\n✓ Migration completed successfully!');
} catch (error) {
await transaction.rollback();
console.error('\n✗ Migration failed:', error.message);
throw error;
}
}
// Main function
async function main() {
const args = parseArgs();
if (args.help) {
showHelp();
return;
}
try {
// Test database connection
await sequelize.authenticate();
console.log('Database connection established.');
if (args.listUsers) {
await listUsers();
return;
}
// Validate arguments
if (!args.userId && !args.username) {
console.error('\nError: You must specify either --userId or --username');
console.log('Use --help for usage information.');
process.exit(1);
}
// Get user ID
let userId = args.userId;
if (args.username) {
const user = await UserModel.findOne({
where: { username: args.username }
});
if (!user) {
console.error(`\nError: User '${args.username}' not found.`);
console.log('Use --list-users to see available users.');
process.exit(1);
}
userId = user.id;
console.log(`Found user '${args.username}' with ID ${userId}`);
} else {
// Verify user exists
const user = await UserModel.findByPk(userId);
if (!user) {
console.error(`\nError: User with ID ${userId} not found.`);
console.log('Use --list-users to see available users.');
process.exit(1);
}
console.log(`Found user '${user.username}' with ID ${userId}`);
}
// Perform migration
await migrateData(userId, args.dryRun);
} catch (error) {
console.error('\nError:', error.message);
process.exit(1);
} finally {
await sequelize.close();
}
}
// Run the script
if (require.main === module) {
main().catch(error => {
console.error('Fatal error:', error);
process.exit(1);
});
}
module.exports = { main, listUsers, migrateData, getStatistics };
+13 -3
View File
@@ -1,5 +1,15 @@
{
"watch": ["back", ".env"],
"watch": [
"back",
".env"
],
"ext": "js,ts,json",
"exec": "ts-node -P tsconfig.back.json ./back/app.ts"
}
"env": {
"NODE_ENV": "development",
"TS_NODE_PROJECT": "./back/tsconfig.json"
},
"verbose": true,
"execMap": {
"ts": "node --require ts-node/register"
}
}
-13
View File
@@ -1,13 +0,0 @@
module.exports = {
apps: [
{
name: 'update',
max_restarts: 10,
kill_timeout: 15000,
wait_ready: true,
listen_timeout: 10000,
time: true,
script: 'static/build/update.js',
},
],
};
+29 -32
View File
@@ -2,19 +2,12 @@
"private": true,
"scripts": {
"start": "concurrently -n w: npm:start:*",
"start:update": "ts-node -P tsconfig.back.json ./back/update.ts",
"start:public": "ts-node -P tsconfig.back.json ./back/public.ts",
"start:rpc": "ts-node -P tsconfig.back.json ./back/schedule/index.ts",
"start:back": "nodemon",
"start:back": "nodemon ./back/app.ts",
"start:front": "max dev",
"build:front": "max build",
"build:back": "tsc -p tsconfig.back.json",
"build:back": "tsc -p back/tsconfig.json",
"panel": "npm run build:back && node static/build/app.js",
"schedule": "npm run build:back && node static/build/schedule/index.js",
"public": "npm run build:back && node static/build/public.js",
"update": "npm run build:back && node static/build/update.js",
"gen:proto": "protoc --experimental_allow_proto3_optional --plugin=./node_modules/.bin/protoc-gen-ts_proto ./back/protos/*.proto --ts_proto_out=./ --ts_proto_opt=outputServices=grpc-js,env=node,esModuleInterop=true",
"gen:api": "python3 -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. ./back/protos/api.proto",
"gen:proto": "protoc --experimental_allow_proto3_optional --plugin=./node_modules/.bin/protoc-gen-ts_proto ./back/protos/*.proto --ts_proto_out=./ --ts_proto_opt=outputServices=grpc-js,env=node,esModuleInterop=true,snakeToCamel=false",
"prettier": "prettier --write '**/*.{js,jsx,tsx,ts,less,md,json}'",
"postinstall": "max setup 2>/dev/null || true",
"test": "umi-test",
@@ -61,12 +54,16 @@
}
},
"dependencies": {
"@grpc/grpc-js": "^1.12.3",
"@grpc/proto-loader": "^0.7.13",
"@bufbuild/protobuf": "^2.10.0",
"@grpc/grpc-js": "^1.14.0",
"@grpc/proto-loader": "^0.8.0",
"@keyv/sqlite": "^4.0.1",
"@otplib/preset-default": "^12.0.1",
"bcrypt": "^6.0.0",
"body-parser": "^1.20.3",
"celebrate": "^15.0.3",
"chokidar": "^4.0.1",
"compression": "^1.7.4",
"cors": "^2.8.5",
"cron-parser": "^4.9.0",
"cross-spawn": "^7.0.6",
@@ -76,51 +73,49 @@
"express-jwt": "^8.4.1",
"express-rate-limit": "^7.4.1",
"express-urlrewrite": "^2.0.3",
"form-data": "^4.0.0",
"got": "^11.8.2",
"helmet": "^8.1.0",
"hpagent": "^1.2.0",
"http-proxy-middleware": "^3.0.3",
"iconv-lite": "^0.6.3",
"ip2region": "2.3.0",
"js-yaml": "^4.1.0",
"jsonwebtoken": "^9.0.2",
"keyv": "^5.2.3",
"lodash": "^4.17.21",
"multer": "1.4.5-lts.1",
"node-schedule": "^2.1.0",
"nodemailer": "^6.9.16",
"p-queue-cjs": "7.3.4",
"protobufjs": "^7.4.0",
"pstree.remy": "^1.1.8",
"proper-lockfile": "^4.1.2",
"ps-tree": "^1.2.0",
"reflect-metadata": "^0.2.2",
"request-ip": "3.3.0",
"sequelize": "^6.37.5",
"serve-handler": "^6.1.6",
"sockjs": "^0.3.24",
"sqlite3": "git+https://github.com/whyour/node-sqlite3.git#v1.0.3",
"toad-scheduler": "^3.0.1",
"typedi": "^0.10.0",
"undici": "^7.9.0",
"uuid": "^11.0.3",
"winston": "^3.17.0",
"winston-daily-rotate-file": "^5.0.0",
"request-ip": "3.3.0",
"ip2region": "2.3.0",
"keyv": "^5.2.3",
"@keyv/sqlite": "^4.0.1",
"proper-lockfile": "^4.1.2"
"winston-daily-rotate-file": "^5.0.0"
},
"devDependencies": {
"moment": "2.30.1",
"@ant-design/icons": "^5.0.1",
"@ant-design/pro-layout": "6.38.22",
"@codemirror/view": "^6.34.1",
"@codemirror/state": "^6.4.1",
"@codemirror/view": "^6.34.1",
"@monaco-editor/react": "4.2.1",
"@react-hook/resize-observer": "^2.0.2",
"react-router-dom": "6.26.1",
"@types/bcrypt": "^6.0.0",
"@types/body-parser": "^1.19.2",
"@types/compression": "^1.7.2",
"@types/cors": "^2.8.12",
"@types/cross-spawn": "^6.0.2",
"@types/express": "^4.17.13",
"@types/express-jwt": "^6.0.4",
"@types/file-saver": "2.0.2",
"@types/helmet": "^4.0.0",
"@types/js-yaml": "^4.0.5",
"@types/jsonwebtoken": "^8.5.8",
"@types/lodash": "^4.14.185",
@@ -128,16 +123,17 @@
"@types/node": "^17.0.21",
"@types/node-schedule": "^1.3.2",
"@types/nodemailer": "^6.4.4",
"@types/proper-lockfile": "^4.1.4",
"@types/ps-tree": "^1.1.6",
"@types/qrcode.react": "^1.0.2",
"@types/react": "^18.0.20",
"@types/react-copy-to-clipboard": "^5.0.4",
"@types/react-dom": "^18.0.6",
"@types/request-ip": "0.0.41",
"@types/serve-handler": "^6.1.1",
"@types/sockjs": "^0.3.33",
"@types/sockjs-client": "^1.5.1",
"@types/uuid": "^8.3.4",
"@types/request-ip": "0.0.41",
"@types/proper-lockfile": "^4.1.4",
"@uiw/codemirror-extensions-langs": "^4.21.9",
"@uiw/react-codemirror": "^4.21.9",
"@umijs/max": "^4.4.4",
@@ -149,9 +145,9 @@
"axios": "^1.4.0",
"compression-webpack-plugin": "9.2.0",
"concurrently": "^7.0.0",
"react-hotkeys-hook": "^4.6.1",
"file-saver": "2.0.2",
"lint-staged": "^13.0.3",
"moment": "2.30.1",
"monaco-editor": "0.33.0",
"nodemon": "^3.0.1",
"prettier": "^2.5.1",
@@ -167,15 +163,16 @@
"react-dnd": "^16.0.1",
"react-dnd-html5-backend": "^16.0.1",
"react-dom": "18.3.1",
"react-hotkeys-hook": "^4.6.1",
"react-intl-universal": "^2.12.0",
"react-router-dom": "6.26.1",
"react-split-pane": "^0.1.92",
"sockjs-client": "^1.6.0",
"ts-node": "^10.9.2",
"ts-proto": "^1.146.0",
"ts-proto": "^2.6.1",
"tslib": "^2.4.0",
"typescript": "5.2.2",
"vh-check": "^2.0.5",
"virtualizedtableforantd4": "1.3.0",
"yorkie": "^2.0.0"
"virtualizedtableforantd4": "1.3.0"
}
}
+173 -351
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -226,9 +226,17 @@ export QMSG_TYPE=""
## ntfy_url 填写ntfy地址,如https://ntfy.sh
## ntfy_topic 填写ntfy的消息应用topic
## ntfy_priority 填写推送消息优先级,默认为3
## ntfy_token 填写推送token,可选
## ntfy_username 填写推送用户名称,可选
## ntfy_password 填写推送用户密码,可选
## ntfy_actions 填写推送用户动作,可选
export NTFY_URL=""
export NTFY_TOPIC=""
export NTFY_PRIORITY="3"
export NTFY_TOKEN=""
export NTFY_USERNAME=""
export NTFY_PASSWORD=""
export NTFY_ACTIONS=""
## 21. wxPusher
## 官方文档: https://wxpusher.zjiecode.com/docs/
+74 -32
View File
@@ -1,7 +1,42 @@
const querystring = require('node:querystring');
const got = require('got');
const { request: undiciRequest, ProxyAgent, FormData } = require('undici');
const timeout = 15000;
async function request(url, options = {}) {
const { json, form, body, headers = {}, ...rest } = options;
const finalHeaders = { ...headers };
let finalBody = body;
if (json) {
finalHeaders['content-type'] = 'application/json';
finalBody = JSON.stringify(json);
} else if (form) {
finalBody = form;
delete finalHeaders['content-type'];
}
return undiciRequest(url, {
headers: finalHeaders,
body: finalBody,
...rest,
});
}
function post(url, options = {}) {
return request(url, { ...options, method: 'POST' });
}
function get(url, options = {}) {
return request(url, { ...options, method: 'GET' });
}
const httpClient = {
request,
post,
get,
};
const push_config = {
HITOKOTO: true, // 启用一言(随机句子)
@@ -84,7 +119,8 @@ const push_config = {
AIBOTK_NAME: '', // 智能微秘书 发送群名 或者好友昵称和type要对应好
SMTP_SERVICE: '', // 邮箱服务名称,比如 126、163、Gmail、QQ 等,支持列表 https://github.com/nodemailer/nodemailer/blob/master/lib/well-known/services.json
SMTP_EMAIL: '', // SMTP 发件邮箱,通知将会由自己发给自己
SMTP_EMAIL: '', // SMTP 发件邮箱
SMTP_TO: '', // SMTP 收件邮箱,默认通知将会发给发件邮箱
SMTP_PASSWORD: '', // SMTP 登录密码,也可能为特殊口令,视具体邮件服务商说明而定
SMTP_NAME: '', // SMTP 收发件人姓名,可随意填写
@@ -104,6 +140,10 @@ const push_config = {
NTFY_URL: '', // ntfy地址,如https://ntfy.sh,默认为https://ntfy.sh
NTFY_TOPIC: '', // ntfy的消息应用topic
NTFY_PRIORITY: '3', // 推送消息优先级,默认为3
NTFY_TOKEN: '', // 推送token,可选
NTFY_USERNAME: '', // 推送用户名称,可选
NTFY_PASSWORD: '', // 推送用户密码,可选
NTFY_ACTIONS: '', // 推送用户动作,可选
// 官方文档: https://wxpusher.zjiecode.com/docs/
// 管理后台: https://wxpusher.zjiecode.com/admin/
@@ -122,9 +162,9 @@ for (const key in push_config) {
const $ = {
post: (params, callback) => {
const { url, ...others } = params;
got.post(url, others).then(
(res) => {
let body = res.body;
httpClient.post(url, others).then(
async (res) => {
let body = await res.body.text();
try {
body = JSON.parse(body);
} catch (error) {}
@@ -137,9 +177,9 @@ const $ = {
},
get: (params, callback) => {
const { url, ...others } = params;
got.get(url, others).then(
(res) => {
let body = res.body;
httpClient.get(url, others).then(
async (res) => {
let body = await res.body.text();
try {
body = JSON.parse(body);
} catch (error) {}
@@ -155,8 +195,8 @@ const $ = {
async function one() {
const url = 'https://v1.hitokoto.cn/';
const res = await got.get(url);
const body = JSON.parse(res.body);
const res = await httpClient.request(url);
const body = await res.body.json();
return `${body.hitokoto} ----${body.from}`;
}
@@ -441,21 +481,11 @@ function tgBotNotify(text, desp) {
timeout,
};
if (TG_PROXY_HOST && TG_PROXY_PORT) {
const { HttpProxyAgent, HttpsProxyAgent } = require('hpagent');
const _options = {
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 256,
maxFreeSockets: 256,
proxy: `http://${TG_PROXY_AUTH}${TG_PROXY_HOST}:${TG_PROXY_PORT}`,
};
const httpAgent = new HttpProxyAgent(_options);
const httpsAgent = new HttpsProxyAgent(_options);
const agent = {
http: httpAgent,
https: httpsAgent,
};
options.agent = agent;
let agent;
agent = new ProxyAgent({
uri: `http://${TG_PROXY_AUTH}${TG_PROXY_HOST}:${TG_PROXY_PORT}`,
});
options.dispatcher = agent;
}
$.post(options, (err, resp, data) => {
try {
@@ -993,7 +1023,8 @@ function fsBotNotify(text, desp) {
}
async function smtpNotify(text, desp) {
const { SMTP_EMAIL, SMTP_PASSWORD, SMTP_SERVICE, SMTP_NAME } = push_config;
const { SMTP_EMAIL, SMTP_TO, SMTP_PASSWORD, SMTP_SERVICE, SMTP_NAME } =
push_config;
if (![SMTP_EMAIL, SMTP_PASSWORD].every(Boolean) || !SMTP_SERVICE) {
return;
}
@@ -1011,7 +1042,7 @@ async function smtpNotify(text, desp) {
const addr = SMTP_NAME ? `"${SMTP_NAME}" <${SMTP_EMAIL}>` : SMTP_EMAIL;
const info = await transporter.sendMail({
from: addr,
to: addr,
to: SMTP_TO ? SMTP_TO.split(';') : addr,
subject: text,
html: `${desp.replace(/\n/g, '<br/>')}`,
});
@@ -1207,17 +1238,18 @@ function webhookNotify(text, desp) {
'$title',
encodeURIComponent(text),
).replaceAll('$content', encodeURIComponent(desp));
got(formatUrl, options).then((resp) => {
httpClient.request(formatUrl, options).then(async (resp) => {
const body = await resp.body.text();
try {
if (resp.statusCode !== 200) {
console.log(`自定义发送通知消息失败😞 ${resp.body}\n`);
console.log(`自定义发送通知消息失败😞 ${body}\n`);
} else {
console.log(`自定义发送通知消息成功🎉 ${resp.body}\n`);
console.log(`自定义发送通知消息成功🎉 ${body}\n`);
}
} catch (e) {
$.logErr(e, resp);
} finally {
resolve(resp.body);
resolve(body);
}
});
});
@@ -1230,7 +1262,7 @@ function ntfyNotify(text, desp) {
}
return new Promise((resolve) => {
const { NTFY_URL, NTFY_TOPIC, NTFY_PRIORITY } = push_config;
const { NTFY_URL, NTFY_TOPIC, NTFY_PRIORITY, NTFY_TOKEN, NTFY_USERNAME, NTFY_PASSWORD, NTFY_ACTIONS } = push_config;
if (NTFY_TOPIC) {
const options = {
url: `${NTFY_URL || 'https://ntfy.sh'}/${NTFY_TOPIC}`,
@@ -1238,9 +1270,19 @@ function ntfyNotify(text, desp) {
headers: {
Title: `${encodeRFC2047(text)}`,
Priority: NTFY_PRIORITY || '3',
Icon: 'https://qn.whyour.cn/logo.png',
},
timeout,
};
if (NTFY_TOKEN) {
options.headers['Authorization'] = `Bearer ${NTFY_TOKEN}`;
} else if (NTFY_USERNAME && NTFY_PASSWORD) {
options.headers['Authorization'] = `Basic ${Buffer.from(`${NTFY_USERNAME}:${NTFY_PASSWORD}`).toString('base64')}`;
}
if (NTFY_ACTIONS) {
options.headers['Actions'] = encodeRFC2047(NTFY_ACTIONS);
}
$.post(options, (err, resp, data) => {
try {
if (err) {
+12 -1
View File
@@ -126,6 +126,10 @@ push_config = {
'NTFY_URL': '', # ntfy地址,如https://ntfy.sh
'NTFY_TOPIC': '', # ntfy的消息应用topic
'NTFY_PRIORITY':'3', # 推送消息优先级,默认为3
'NTFY_TOKEN': '', # 推送token,可选
'NTFY_USERNAME': '', # 推送用户名称,可选
'NTFY_PASSWORD': '', # 推送用户密码,可选
'NTFY_ACTIONS': '', # 推送用户动作,可选
'WXPUSHER_APP_TOKEN': '', # wxpusher 的 appToken 官方文档: https://wxpusher.zjiecode.com/docs/ 管理后台: https://wxpusher.zjiecode.com/admin/
'WXPUSHER_TOPIC_IDS': '', # wxpusher 的 主题ID,多个用英文分号;分隔 topic_ids 与 uids 至少配置一个才行
@@ -806,7 +810,14 @@ def ntfy(title: str, content: str) -> None:
encoded_title = encode_rfc2047(title)
data = content.encode(encoding="utf-8")
headers = {"Title": encoded_title, "Priority": priority} # 使用编码后的 title
headers = {"Title": encoded_title, "Priority": priority, "Icon": "https://qn.whyour.cn/logo.png"} # 使用编码后的 title
if push_config.get("NTFY_TOKEN"):
headers['Authorization'] = "Bearer " + push_config.get("NTFY_TOKEN")
elif push_config.get("NTFY_USERNAME") and push_config.get("NTFY_PASSWORD"):
authStr = push_config.get("NTFY_USERNAME") + ":" + push_config.get("NTFY_PASSWORD")
headers['Authorization'] = "Basic " + base64.b64encode(authStr.encode('utf-8')).decode('utf-8')
if push_config.get("NTFY_ACTIONS"):
headers['Actions'] = encode_rfc2047(push_config.get("NTFY_ACTIONS"))
url = push_config.get("NTFY_URL") + "/" + push_config.get("NTFY_TOPIC")
response = requests.post(url, data=data, headers=headers)
+8 -48
View File
@@ -41,14 +41,9 @@ add_cron_api() {
fi
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/crons?t=$currentTimeStamp" \
-H "Accept: application/json" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--data-raw "{\"name\":\"${name//\"/\\\"}\",\"command\":\"${command//\"/\\\"}\",\"schedule\":\"$schedule\",\"sub_id\":$sub_id}" \
--compressed
)
@@ -76,15 +71,10 @@ update_cron_api() {
fi
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/crons?t=$currentTimeStamp" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
-X 'PUT' \
-H "Accept: application/json" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--data-raw "{\"name\":\"${name//\"/\\\"}\",\"command\":\"${command//\"/\\\"}\",\"schedule\":\"$schedule\",\"id\":\"$id\"}" \
--compressed
)
@@ -108,15 +98,10 @@ update_cron_command_api() {
fi
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/crons?t=$currentTimeStamp" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
-X 'PUT' \
-H "Accept: application/json" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--data-raw "{\"command\":\"${command//\"/\\\"}\",\"id\":\"$id\"}" \
--compressed
)
@@ -133,15 +118,10 @@ del_cron_api() {
local ids="$1"
local currentTimeStamp=$(date +%s)
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/crons?t=$currentTimeStamp" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons?t=$currentTimeStamp" \
-X 'DELETE' \
-H "Accept: application/json" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--data-raw "[$ids]" \
--compressed
)
@@ -163,15 +143,10 @@ update_cron() {
local runningTime="${6:-0}"
local currentTimeStamp=$(date +%s)
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/crons/status?t=$currentTimeStamp" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons/status?t=$currentTimeStamp" \
-X 'PUT' \
-H "Accept: application/json" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--data-raw "{\"ids\":[$ids],\"status\":\"$status\",\"pid\":\"$pid\",\"log_path\":\"$logPath\",\"last_execution_time\":$lastExecutingTime,\"last_running_time\":$runningTime}" \
--compressed
)
@@ -190,15 +165,10 @@ notify_api() {
local content="$2"
local currentTimeStamp=$(date +%s)
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/system/notify?t=$currentTimeStamp" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/system/notify?t=$currentTimeStamp" \
-X 'PUT' \
-H "Accept: application/json" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--data-raw "{\"title\":\"${title//\"/\\\"}\",\"content\":\"${content//\"/\\\"}\"}" \
--compressed
)
@@ -215,14 +185,9 @@ find_cron_api() {
local params="$1"
local currentTimeStamp=$(date +%s)
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/crons/detail?$params&t=$currentTimeStamp" \
-H "Accept: application/json" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/crons/detail?$params&t=$currentTimeStamp" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--compressed
)
data=$(echo "$api" | jq -r .data)
@@ -239,15 +204,10 @@ update_auth_config() {
local tip="$2"
local currentTimeStamp=$(date +%s)
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/open/system/auth/reset?t=$currentTimeStamp" \
curl -s --noproxy "*" "http://0.0.0.0:5700/open/system/auth/reset?t=$currentTimeStamp" \
-X 'PUT' \
-H "Accept: application/json" \
-H "Authorization: Bearer ${__ql_token__}" \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" \
-H "Content-Type: application/json;charset=UTF-8" \
-H "Origin: http://0.0.0.0:5700" \
-H "Referer: http://0.0.0.0:5700/crontab" \
-H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" \
--data-raw "{$body}" \
--compressed
)
+3 -22
View File
@@ -20,32 +20,16 @@ copy_dep() {
echo -e "---> 复制一份 $file_notify_js_sample$file_notify_js\n"
cp -fv $file_notify_js_sample $file_notify_js
echo -e "---> 通知文件复制完成\n"
echo -e "---> 2. 复制nginx配置文件\n"
init_nginx
echo -e "---> 配置文件复制完成\n"
}
pm2_log() {
echo -e "---> pm2日志"
local panelOut="/root/.pm2/logs/panel-out.log"
local panelError="/root/.pm2/logs/panel-error.log"
local panelOut="/root/.pm2/logs/qinglong-out.log"
local panelError="/root/.pm2/logs/qinglong-error.log"
tail -n 300 "$panelOut"
tail -n 300 "$panelError"
}
check_nginx() {
local nginxPid=$(ps -eo pid,command | grep nginx | grep -v grep)
echo -e "=====> 检测nginx服务\n$nginxPid"
if [[ $nginxPid ]]; then
echo -e "\n=====> nginx服务正常\n"
nginx -s reload
else
echo -e "\n=====> nginx服务异常,重新启动nginx\n"
nginx -c /etc/nginx/nginx.conf
fi
}
check_ql() {
local api=$(curl -s --noproxy "*" "http://0.0.0.0:5700")
echo -e "\n=====> 检测面板\n\n$api\n"
@@ -58,7 +42,7 @@ check_pm2() {
pm2_log
local currentTimeStamp=$(date +%s)
local api=$(
curl -s --noproxy "*" "http://0.0.0.0:5600/api/system?t=$currentTimeStamp" \
curl -s --noproxy "*" "http://0.0.0.0:5700/api/system?t=$currentTimeStamp" \
-H 'Accept: */*' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36' \
-H 'Referer: http://0.0.0.0:5700/crontab' \
@@ -74,14 +58,11 @@ check_pm2() {
main() {
echo -e "=====> 开始检测"
npm i -g pnpm@8.3.1 pm2 ts-node
patch_version
reset_env
copy_dep
check_ql
check_nginx
check_pm2
reload_update
reload_pm2
echo -e "\n=====> 检测结束\n"
}
+134 -39
View File
@@ -1,45 +1,140 @@
const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');
const { join } = require('path');
const PROTO_PATH = `${process.env.QL_DIR}/back/protos/api.proto`;
const options = {
keepCase: true,
longs: String,
enums: String,
defaults: true,
};
const packageDefinition = protoLoader.loadSync(PROTO_PATH, options);
const apiProto = grpc.loadPackageDefinition(packageDefinition).com.ql.api;
const client = new apiProto.Api(
`0.0.0.0:5500`,
grpc.credentials.createInsecure(),
{ 'grpc.enable_http_proxy': 0 },
);
const promisify = (fn) => {
return (...args) => {
return new Promise((resolve, reject) => {
fn.call(client, ...args, (err, response) => {
if (err) return reject(err);
resolve(response);
});
});
class GrpcClient {
static #config = {
protoPath: join(process.env.QL_DIR, 'back/protos/api.proto'),
serverAddress: '0.0.0.0:5500',
protoOptions: {
keepCase: true,
longs: String,
enums: String,
defaults: true,
},
grpcOptions: {
'grpc.enable_http_proxy': 0,
},
defaultTimeout: 30000,
};
};
const api = {
getEnvs: promisify(client.GetEnvs),
createEnv: promisify(client.CreateEnv),
updateEnv: promisify(client.UpdateEnv),
deleteEnvs: promisify(client.DeleteEnvs),
moveEnv: promisify(client.MoveEnv),
disableEnvs: promisify(client.DisableEnvs),
enableEnvs: promisify(client.EnableEnvs),
updateEnvNames: promisify(client.UpdateEnvNames),
getEnvById: promisify(client.GetEnvById),
systemNotify: promisify(client.SystemNotify),
};
static #methods = [
'getEnvs',
'createEnv',
'updateEnv',
'deleteEnvs',
'moveEnv',
'disableEnvs',
'enableEnvs',
'updateEnvNames',
'getEnvById',
'systemNotify',
'getCronDetail',
'createCron',
'updateCron',
'deleteCrons',
];
module.exports = api;
#client;
#api = {};
constructor() {
this.#initializeClient();
this.#bindMethods();
}
#initializeClient() {
try {
const { protoPath, protoOptions, serverAddress, grpcOptions } =
GrpcClient.#config;
const packageDefinition = protoLoader.loadSync(protoPath, protoOptions);
const apiProto = grpc.loadPackageDefinition(packageDefinition).com.ql.api;
this.#client = new apiProto.Api(
serverAddress,
grpc.credentials.createInsecure(),
grpcOptions,
);
} catch (error) {
console.error('Failed to initialize gRPC client:', error);
process.exit(1);
}
}
#promisifyMethod(methodName) {
const capitalizedMethod =
methodName.charAt(0).toUpperCase() + methodName.slice(1);
const method = this.#client[capitalizedMethod].bind(this.#client);
return async (params = {}) => {
return new Promise((resolve, reject) => {
const metadata = new grpc.Metadata();
const deadline = new Date(
Date.now() + GrpcClient.#config.defaultTimeout,
);
method(params, metadata, { deadline }, (error, response) => {
if (error) {
return reject(error);
}
resolve(response);
});
});
};
}
#bindMethods() {
GrpcClient.#methods.forEach((method) => {
this.#api[method] = this.#promisifyMethod(method);
});
}
getApi() {
return {
...this.#api,
close: this.close.bind(this),
};
}
close() {
if (this.#client) {
this.#client.close();
this.#client = null;
}
}
}
const grpcClient = new GrpcClient();
process.on('SIGTERM', () => {
grpcClient.close();
process.exit(0);
});
process.on('SIGINT', () => {
grpcClient.close();
process.exit(0);
});
process.on('unhandledRejection', (reason, promise) => {
if (reason instanceof Error) {
if (reason.stack) {
const relevantStack = reason.stack
.split('\n')
.filter((line) => {
return (
!line.includes('node:internal') &&
!line.includes('node_modules/@grpc') &&
!line.includes('processTicksAndRejections')
);
})
.join('\n');
console.error(relevantStack);
}
} else {
console.error(reason);
}
});
module.exports = grpcClient.getApi();
+200 -30
View File
@@ -2,7 +2,7 @@ import subprocess
import json
import tempfile
import os
from typing import Dict, List
from typing import Dict, List, TypedDict, Optional
from functools import wraps
@@ -11,16 +11,170 @@ def error_handler(func):
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except json.JSONDecodeError as e:
raise Exception(f"parse json error: {str(e)}")
except subprocess.SubprocessError as e:
raise Exception(f"node process error: {str(e)}")
except TypeError as e:
if "missing" in str(e):
func_name = func.__name__
annotations = func.__annotations__
param_type = next(
(t for name, t in annotations.items() if name != "return"), None
)
if param_type and hasattr(param_type, "__annotations__"):
required_fields = {
k: v
for k, v in param_type.__annotations__.items()
if not getattr(param_type, "__total__", True)
or k in getattr(param_type, "__required_keys__", set())
}
fields_str = ", ".join(
f'"{k}": {v.__name__}' for k, v in required_fields.items()
)
raise Exception(
f"{func_name}() requires a dictionary with parameters: {{{fields_str}}}"
) from None
raise Exception(f"{str(e)}") from None
except Exception as e:
raise Exception(f"unknown error: {str(e)}")
error_msg = str(e)
if "Error:" in error_msg:
error_msg = error_msg.split("Error:")[-1].split("\n")[0].strip()
raise Exception(f"{error_msg}") from None
return wrapper
class EnvItem(TypedDict, total=False):
id: Optional[int]
name: Optional[str]
value: Optional[str]
remarks: Optional[str]
status: Optional[int]
position: Optional[int]
class GetEnvsParams(TypedDict, total=False):
searchValue: str
class CreateEnvParams(TypedDict):
envs: List[EnvItem]
class UpdateEnvParams(TypedDict):
env: EnvItem
class DeleteEnvsParams(TypedDict):
ids: List[int]
class MoveEnvParams(TypedDict):
id: int
fromIndex: int
toIndex: int
class DisableEnvsParams(TypedDict):
ids: List[int]
class EnableEnvsParams(TypedDict):
ids: List[int]
class UpdateEnvNamesParams(TypedDict):
ids: List[int]
name: str
class GetEnvByIdParams(TypedDict):
id: int
class SystemNotifyParams(TypedDict):
title: str
content: str
class EnvsResponse(TypedDict):
code: int
data: List[EnvItem]
message: Optional[str]
class EnvResponse(TypedDict):
code: int
data: EnvItem
message: Optional[str]
class Response(TypedDict):
code: int
message: Optional[str]
class ExtraScheduleItem(TypedDict, total=False):
schedule: Optional[str]
class CronItem(TypedDict, total=False):
id: Optional[int]
command: Optional[str]
schedule: Optional[str]
name: Optional[str]
labels: List[str]
sub_id: Optional[int]
extra_schedules: List[ExtraScheduleItem]
task_before: Optional[str]
task_after: Optional[str]
status: Optional[int]
log_path: Optional[str]
pid: Optional[int]
last_running_time: Optional[int]
last_execution_time: Optional[int]
class CreateCronParams(TypedDict):
command: str
schedule: str
name: Optional[str]
labels: List[str]
sub_id: Optional[int]
extra_schedules: List[ExtraScheduleItem]
task_before: Optional[str]
task_after: Optional[str]
class UpdateCronParams(TypedDict):
id: int
command: str
schedule: str
name: Optional[str]
labels: List[str]
sub_id: Optional[int]
extra_schedules: List[ExtraScheduleItem]
task_before: Optional[str]
task_after: Optional[str]
class DeleteCronsParams(TypedDict):
ids: List[int]
class CronDetailParams(TypedDict):
log_path: str
class CronsResponse(TypedDict):
code: int
data: List[CronItem]
message: Optional[str]
class CronResponse(TypedDict):
code: int
data: CronItem
message: Optional[str]
class Client:
def __init__(self):
self.temp_dir = tempfile.mkdtemp(prefix="node_client_")
@@ -46,7 +200,8 @@ class Client:
}} catch (error) {{
console.error(JSON.stringify({{
error: error.message,
stack: error.stack
stack: error.stack,
name: error.name
}}));
process.exit(1);
}}
@@ -56,58 +211,73 @@ class Client:
with open(self.temp_script, "w", encoding="utf-8") as f:
f.write(node_code)
try:
result = subprocess.run(
["node", self.temp_script],
capture_output=True,
text=True,
timeout=30,
result = subprocess.run(
["node", self.temp_script],
capture_output=True,
text=True,
timeout=30,
)
if result.returncode != 0:
error_data = json.loads(result.stderr)
raise Exception(
f"{error_data.get('name', 'Error')}: {error_data.get('stack')}"
)
if result.returncode != 0:
error_data = json.loads(result.stderr)
raise Exception(f"{error_data.get('stack')}")
return json.loads(result.stdout)
except subprocess.TimeoutExpired:
raise Exception("node process timeout")
return json.loads(result.stdout)
@error_handler
def getEnvs(self, params: Dict = None) -> Dict:
def getEnvs(self, params: GetEnvsParams = None) -> EnvsResponse:
return self._execute_node("getEnvs", params)
@error_handler
def createEnv(self, data: Dict) -> Dict:
def createEnv(self, data: CreateEnvParams) -> EnvsResponse:
return self._execute_node("createEnv", data)
@error_handler
def updateEnv(self, data: Dict) -> Dict:
def updateEnv(self, data: UpdateEnvParams) -> EnvResponse:
return self._execute_node("updateEnv", data)
@error_handler
def deleteEnvs(self, data: Dict) -> Dict:
def deleteEnvs(self, data: DeleteEnvsParams) -> Response:
return self._execute_node("deleteEnvs", data)
@error_handler
def moveEnv(self, data: Dict) -> Dict:
def moveEnv(self, data: MoveEnvParams) -> EnvResponse:
return self._execute_node("moveEnv", data)
@error_handler
def disableEnvs(self, data: Dict) -> Dict:
def disableEnvs(self, data: DisableEnvsParams) -> Response:
return self._execute_node("disableEnvs", data)
@error_handler
def enableEnvs(self, data: Dict) -> Dict:
def enableEnvs(self, data: EnableEnvsParams) -> Response:
return self._execute_node("enableEnvs", data)
@error_handler
def updateEnvNames(self, data: Dict) -> Dict:
def updateEnvNames(self, data: UpdateEnvNamesParams) -> Response:
return self._execute_node("updateEnvNames", data)
@error_handler
def getEnvById(self, data: Dict) -> Dict:
def getEnvById(self, data: GetEnvByIdParams) -> EnvResponse:
return self._execute_node("getEnvById", data)
@error_handler
def systemNotify(self, data: Dict) -> Dict:
def systemNotify(self, data: SystemNotifyParams) -> Response:
return self._execute_node("systemNotify", data)
@error_handler
def getCronDetail(self, data: CronDetailParams) -> CronResponse:
return self._execute_node("getCronDetail", data)
@error_handler
def createCron(self, data: CreateCronParams) -> CronResponse:
return self._execute_node("createCron", data)
@error_handler
def updateCron(self, data: UpdateCronParams) -> CronResponse:
return self._execute_node("updateCron", data)
@error_handler
def deleteCrons(self, data: DeleteCronsParams) -> Response:
return self._execute_node("deleteCrons", data)

Some files were not shown because too many files have changed in this diff Show More