qinglong/docs/PROJECT_ARCHITECTURE.md

Project Architecture Guide

This document is written for AI coding agents and maintainers who need to understand and modify this project safely. It focuses on where behavior lives, how the application starts, and which files are usually involved for common changes.

Project Summary

Qinglong is a timed task management platform. It provides a web admin panel for managing cron jobs, scripts, environment variables, subscriptions, dependencies, logs, configuration files, and system settings.

The repository is organized as a full-stack TypeScript application:

• src/: frontend admin panel, built with Umi Max, React, Ant Design, and Ant Design Pro Layout.
• back/: backend application, built with Express, TypeScript, typedi, Sequelize, SQLite, gRPC, and worker processes.
• shell/: runtime shell scripts used to execute tasks and preload task environments.
• data/: local runtime data, including scripts, logs, configs, SQLite database, uploaded files, and cloned repositories.
• static/: built frontend and backend artifacts.
• docker/: Docker images, compose file, and entrypoint.
• sample/: sample scripts and default config templates.

High-Level Runtime Flow

Browser
  -> src/pages/*
  -> src/utils/http.tsx
  -> /api/*
  -> back/api/*
  -> back/services/*
  -> back/data/* Sequelize models
  -> data/db/database.sqlite

Cron/task execution
  -> back/services/cron.ts
  -> shell/task.sh or shell/otask.sh
  -> data/scripts/*
  -> data/log/*

Frontend assets in production
  -> static/dist/*
  -> served by back/loaders/express.ts

Main Startup Path

Development starts from package.json:

pnpm start

This runs:

• start:back: nodemon ./back/app.ts
• start:front: max dev

Backend startup begins in back/app.ts.

Important details:

• The backend uses Node cluster.
• The primary process initializes the database first.
• A gRPC worker starts before the HTTP worker.
• The HTTP worker starts Express and serves API routes plus frontend static files.
• If the gRPC worker restarts, the HTTP worker is asked to re-register cron jobs.

Production-style backend output is generated by:

pnpm run build:back

The compiled backend is placed under static/build.

Frontend output is generated by:

pnpm run build:front

The compiled frontend is placed under static/dist.

Backend Architecture

Entry Point

• back/app.ts

Responsibilities:

• Creates the Express application.
• Starts primary/worker process logic.
• Initializes database in the primary process.
• Starts gRPC and HTTP workers.
• Handles graceful shutdown.
• Re-registers cron jobs after gRPC worker recovery.

Loaders

• back/loaders/app.ts
• back/loaders/express.ts
• back/loaders/db.ts
• back/loaders/depInjector.ts
• back/loaders/initData.ts
• back/loaders/initFile.ts
• back/loaders/initTask.ts
• back/loaders/server.ts
• back/loaders/sock.ts

Loader responsibilities:

• Register dependency injection bindings.
• Sync Sequelize models.
• Initialize files and default data.
• Initialize scheduled tasks.
• Configure Express middleware.
• Register routes.
• Attach socket/server behavior.

back/loaders/express.ts is the main HTTP middleware and routing setup. It handles:

• CORS.
• Helmet.
• body parser.
• static frontend serving.
• JWT validation.
• token validation against shared auth state.
• /open/* rewrite to /api/*.
• route mounting through back/api/index.ts.
• frontend fallback to static/dist/index.html.
• API error handling.

API Routes

• back/api/index.ts

This file registers all API modules:

• user.ts: login, initialization, authentication-related user endpoints.
• env.ts: environment variable endpoints.
• config.ts: config file endpoints.
• log.ts: log endpoints.
• cron.ts: cron/task endpoints.
• script.ts: script file endpoints.
• open.ts: open API/app token endpoints.
• dependence.ts: dependency management endpoints.
• system.ts: system information/settings endpoints.
• subscription.ts: subscription endpoints.
• update.ts: update/check endpoints.
• health.ts: health check endpoints.

Route files should stay thin. They should validate input, get a service from typedi's Container, call the service, and return { code, data, message } style responses.

Services

• back/services/*

Services contain most business logic. Common examples:

• cron.ts: create/update/delete/run cron jobs, generate crontab data, manage logs, call scheduler client.
• env.ts: manage environment variables.
• config.ts: read/write config files.
• script.ts: manage script files.
• subscription.ts: manage script subscriptions and repository pulls.
• dependence.ts: install/manage runtime dependencies.
• system.ts: system info and settings.
• notify.ts: notification behavior.
• sock.ts: socket/log stream behavior.
• grpc.ts: gRPC server lifecycle.
• http.ts: HTTP server lifecycle.

When changing backend behavior, first find the API route, then follow it into the matching service. In most cases, the service is the right place for behavioral changes.

Data Models

• back/data/index.ts
• back/data/*.ts

The backend uses Sequelize with SQLite. Database storage is configured in back/data/index.ts:

data/db/database.sqlite

Common model files:

• cron.ts: cron job model.
• cronView.ts: saved cron table views.
• env.ts: environment variable model.
• dependence.ts: dependency model.
• open.ts: open API app/token model.
• subscription.ts: subscription model.
• system.ts: system settings model.
• notify.ts: notification-related data.

Model sync and simple column migrations are currently handled in back/loaders/db.ts.

Configuration

• back/config/index.ts

This is the central runtime config file. It reads .env, establishes QL_DIR, and defines important paths:

• dataPath: runtime data root.
• configPath: config files.
• scriptPath: user scripts.
• repoPath: subscription repositories.
• logPath: task logs.
• dbPath: SQLite database location.
• uploadPath: uploaded files.
• shellPath: shell runtime scripts.
• preloadPath: JS/Python/Shell preload files.

Before hardcoding paths, check back/config/index.ts.

Scheduling And gRPC

• back/schedule/*
• back/protos/*
• back/services/grpc.ts

The project has two scheduling paths:

• Standard crontab-style tasks are persisted and written through backend cron logic.
• Node/gRPC scheduler logic handles cases such as second-level cron expressions or additional schedules.

back/services/cron.ts decides whether a task needs the Node scheduler using schedule shape and extra_schedules.

Shared Backend Utilities

• back/shared/*
• back/config/util.ts
• back/config/share.ts
• back/config/http.ts

Use these before adding new global helpers. Existing shared code includes:

• auth helpers.
• shared store.
• log stream manager.
• task runner helpers.
• concurrency limits.
• file locking utilities.
• HTTP/proxy helpers.

Frontend Architecture

Umi Config

• .umirc.ts

Important behavior:

• Dev server proxies API requests to http://127.0.0.1:5700/.
• Frontend build output is static/dist.
• Runtime env script is loaded from ./api/env.js.
• QlBaseUrl affects frontend public path and routing base.

App Initialization

• src/app.ts

Responsibilities:

• Load Chinese and English locale JSON.
• Determine locale from URL/cookie/localStorage.
• Set Umi locale.
• Apply QlBaseUrl as public path and router basename.

Layout And Routes

• src/layouts/defaultProps.tsx
• src/layouts/index.tsx

defaultProps.tsx defines the main route/menu list. If adding a new page visible in the sidebar, update this file.

Current major pages:

• src/pages/crontab: timed task management.
• src/pages/subscription: subscription management.
• src/pages/env: environment variables.
• src/pages/config: config files.
• src/pages/script: script management.
• src/pages/dependence: dependency management.
• src/pages/log: log management.
• src/pages/diff: diff tool.
• src/pages/setting: system settings.
• src/pages/login: login.
• src/pages/initialization: first-run initialization.
• src/pages/error: error page.

Frontend Utilities

• src/utils/http.tsx: API request helper.
• src/utils/websocket.ts: socket connection behavior.
• src/utils/config.ts: frontend config helpers.
• src/utils/const.ts: constants.
• src/utils/date.ts: date formatting helpers.
• src/utils/init.ts: initialization helpers.
• src/utils/codemirror/*: CodeMirror integration.
• src/utils/monaco/*: Monaco integration.

When changing a page's API behavior, inspect both the page file and src/utils/http.tsx.

Components And Styling

• src/components/*: reusable UI components.
• src/pages/**/index.less: page-level styles.
• src/pages/script/index.module.less and src/pages/log/index.module.less: CSS module styles.
• src/assets/fonts/*: bundled fonts.
• src/locales/*.json: i18n text.

Follow the existing Ant Design and Ant Design Pro patterns when modifying UI.

Shell Runtime

• shell/task.sh: task execution path.
• shell/otask.sh: alternate/manual task execution path.
• shell/api.sh: shell-side API helpers.
• shell/env.sh: environment setup.
• shell/check.sh: runtime check helpers.
• shell/update.sh: update helpers.
• shell/rmlog.sh: log cleanup.
• shell/share.sh: shared shell helpers.
• shell/preload/*: preload files injected into JS/Python/Shell task environments.

The backend often coordinates task execution, but the actual user script process environment is shaped by files in shell/.

Runtime Data Directory

• data/

This directory is runtime state, not just source code. Be careful when modifying or deleting files here.

Important subdirectories:

• data/db: SQLite database.
• data/config: generated and user-edited config files.
• data/scripts: user scripts.
• data/repo: cloned subscription repositories.
• data/log: task logs.
• data/upload: uploaded files.
• data/syslog: system logs.
• data/ssh.d: SSH-related runtime data.
• data/dep_cache: dependency cache, when present.

Many bugs that appear as "backend logic" may involve state stored under data/.

Docker And Release Files

• docker/Dockerfile
• docker/310.Dockerfile
• docker/docker-compose.yml
• docker/docker-entrypoint.sh
• ecosystem.config.js
• version.yaml

Use these when changing deployment, container startup, PM2 behavior, or release metadata.

Common Modification Map

Add Or Modify A Backend API

Typical files:

1. Add or update route in back/api/<module>.ts.
2. Add or update service logic in back/services/<module>.ts.
3. Add or update model in back/data/<module>.ts if persistence changes.
4. Add validation with celebrate/Joi near the route.
5. Update frontend caller in src/pages/** or src/utils/**.

Add A New Frontend Page

Typical files:

1. Create src/pages/<page>/index.tsx.
2. Add styles in src/pages/<page>/index.less if needed.
3. Register route/menu in src/layouts/defaultProps.tsx.
4. Add locale strings in src/locales/zh-CN.json and src/locales/en-US.json.
5. Add API calls through the existing request helper.

Change Cron/Task Behavior

Start with:

• back/api/cron.ts
• back/services/cron.ts
• back/schedule/*
• shell/task.sh
• shell/otask.sh
• shell/preload/*

Also inspect:

• back/data/cron.ts
• back/validation/schedule.ts
• data/config/crontab.list
• data/log/*

Change Environment Variable Behavior

Start with:

• back/api/env.ts
• back/services/env.ts
• back/data/env.ts
• src/pages/env/index.tsx

Also inspect:

• shell/preload/env.sh
• shell/preload/env.js
• shell/preload/env.py

Change Script Management

Start with:

• back/api/script.ts
• back/services/script.ts
• src/pages/script/index.tsx
• data/scripts/*

Change Login/Auth/Security

Start with:

• back/api/user.ts
• back/services/user.ts
• back/shared/auth.ts
• back/shared/store.ts
• back/loaders/express.ts
• back/token.ts
• src/pages/login/index.tsx
• src/pages/initialization/index.tsx

Be careful with:

• JWT behavior.
• open API token behavior.
• first-run initialization.
• platform-specific session limits.

Change Subscription Behavior

Start with:

• back/api/subscription.ts
• back/services/subscription.ts
• back/data/subscription.ts
• src/pages/subscription/index.tsx
• data/repo/*

Change Dependency Management

Start with:

• back/api/dependence.ts
• back/services/dependence.ts
• back/data/dependence.ts
• src/pages/dependence/index.tsx
• data/deps
• data/dep_cache

Change Logs Or Live Log Streaming

Start with:

• back/api/log.ts
• back/services/log.ts
• back/services/sock.ts
• back/shared/logStreamManager.ts
• src/pages/log/index.tsx
• src/components/terminal.tsx
• data/log/*

Coding Conventions

Backend:

• Prefer adding business logic to services, not route files.
• Use typedi services consistently.
• Use existing config paths from back/config/index.ts.
• Return API responses in the existing { code, data, message } shape.
• Use existing utilities before adding new helpers.
• Preserve current SQLite/Sequelize style unless doing a larger data-layer refactor.

Frontend:

• Follow existing Umi/React/Ant Design patterns.
• Keep route/menu changes in src/layouts/defaultProps.tsx.
• Use existing request/WebSocket helpers.
• Add or update locale strings for visible UI text.
• Keep page-specific styles near the page.

Shell/runtime:

• Treat shell/ as part of production behavior.
• Test task execution changes with realistic scripts when possible.
• Be careful with path quoting and environment variable propagation.

Data:

• Treat data/ as mutable runtime state.
• Do not delete runtime state unless explicitly requested.
• Schema changes should account for existing SQLite databases.

Suggested First Steps For AI Agents

When asked to modify behavior:

1. Identify whether the change is frontend, backend, shell runtime, data model, or deployment.
2. Search by feature name in src/pages, back/api, and back/services.
3. Read the route file and matching service before editing.
4. If persistence is involved, read the matching back/data model and back/loaders/db.ts.
5. If task execution is involved, inspect shell/ and back/services/cron.ts.
6. Make the smallest scoped change that matches existing patterns.
7. Run the most relevant check:
   • pnpm run build:back for backend TypeScript changes.
   • pnpm run build:front for frontend build changes.
   • targeted manual task/API checks for shell and scheduler changes.

Quick Directory Reference

.
├── back/                 Backend TypeScript application
│   ├── api/              Express route modules
│   ├── config/           Runtime config, paths, constants, helpers
│   ├── data/             Sequelize models and SQLite connection
│   ├── loaders/          Startup initialization and Express setup
│   ├── middlewares/      Express middlewares
│   ├── protos/           gRPC proto files and generated TS
│   ├── schedule/         Scheduler/gRPC client helpers
│   ├── services/         Business logic services
│   ├── shared/           Shared backend utilities
│   └── validation/       Joi validation schemas
├── src/                  Frontend Umi/React application
│   ├── assets/           Fonts and static frontend assets
│   ├── components/       Shared UI components
│   ├── hooks/            Frontend hooks
│   ├── layouts/          Main layout and menu route config
│   ├── locales/          i18n JSON
│   ├── pages/            Feature pages
│   └── utils/            HTTP, WebSocket, editor, date, and config utilities
├── shell/                Task runtime shell scripts and preload files
├── data/                 Runtime state: db, logs, scripts, repos, configs
├── docker/               Docker build and compose files
├── sample/               Sample scripts and default config templates
├── static/               Built frontend/backend artifacts
└── docs/                 Project documentation