fix: 完善查询agent时间与检索契约

增加调试与诊断
This commit is contained in:
Wxw-Gu
2026-09-10 17:45:12 +08:00
parent 376ea79ff6
commit fe8619d56d
18 changed files with 1210 additions and 84 deletions
+8 -4
View File
@@ -14,10 +14,14 @@ resources/connectors/wechat-personal/
.omc .omc
.codex/ .codex/
skills-lock.json skills-lock.json
AGENTS.md *__screenshots__
/AGENTS.md
/CLAUDE.md
/.claude/
/.workbuddy/
/.ai-local/
findings.md findings.md
progress.md progress.md
task_plan.md task_plan.md
.agents
*__screenshots__
.ai-local
+1
View File
@@ -48,6 +48,7 @@ Agent Hub 让微信机器人调用本机 TraceMemo;Reader Skill / Local HTTP A
## 开发文档 ## 开发文档
- [开发、测试与构建](./development/overview.md) - [开发、测试与构建](./development/overview.md)
- [Query Agent POC(开发测试入口)](./development/query-agent-poc.md)
- [本地启动排障](./development/local-startup-troubleshooting.md) - [本地启动排障](./development/local-startup-troubleshooting.md)
- [macOS 数据访问说明](./platform/macos.md) - [macOS 数据访问说明](./platform/macos.md)
- [关闭 SIP 教程](./mac-disable-sip.md) - [关闭 SIP 教程](./mac-disable-sip.md)
-27
View File
@@ -1,27 +0,0 @@
# Query Agent POC
这是独立的开发测试入口,不会修改生产“问问微信”执行链。
先启动 TraceMemo,并在 API Center 开启 Local HTTP API。然后在仓库根目录运行 例:
```bash
pnpm poc:query-agent -- "我和BOBO第一次聊了什么"
```
也可以直接传入其他自然语言问题:
```bash
pnpm poc:query-agent -- "BOBO上个月有没有给我发过文件"
```
POC 使用设置页当前默认 AI Provider、模型、Base URL 和安全存储中的 API Key。Local Query API 仍使用现有 Bearer Token;POC 输出不会打印 Token、API Key、数据库路径或内部消息 ID。
输出为 JSON,包含:
- `question`、`provider`、`model`
- `modelCallCount`、`toolCallCount`
- `firstModelMs`、`toolTotalMs`、`finalModelMs`、`totalMs`
- 每次工具调用的名称、脱敏参数、耗时、状态和结果数量
- 最终 `answer` 或错误信息
工具调用最多 5 次,只允许 `query_messages`、`search_messages`、`message_context`、`conversation_overview`。未配置 AI Provider、Local Query API 未启动或当前 Provider 协议不支持 tools 时,POC 会直接返回错误,不会回退到另一套模型配置。
+1 -1
View File
@@ -52,7 +52,7 @@ pnpm test:e2e:build
```bash ```bash
git diff --check git diff --check
rg -n "v2\.1\.7|TraceMemo|迹忆|mcpServers|无鉴权" README.md docs --glob '*.md' --glob '!DOCUMENTATION_AUDIT.md' --glob '!development/overview.md' rg -n "v2\.1\.7|TraceMemo|迹忆|mcpServers|无鉴权" README.md docs --glob '*.md' --glob '!development/overview.md'
``` ```
历史迁移说明可以出现旧版本号;正式使用指南不要把过时版本写成当前版本。负向澄清“6131 不是 MCP Server”可以保留,以防用户照抄错误配置。 历史迁移说明可以出现旧版本号;正式使用指南不要把过时版本写成当前版本。负向澄清“6131 不是 MCP Server”可以保留,以防用户照抄错误配置。
+84
View File
@@ -0,0 +1,84 @@
# Query Agent POC
这是独立的开发测试入口,不会修改生产“问问微信”执行链。
先启动 TraceMemo,并在 API Center 开启 Local HTTP API。然后在仓库根目录运行:
```bash
pnpm poc:query-agent "我和BOBO第一次聊了什么"
```
## 两个入口
| 命令 | 行为 | 何时用 |
| --- | --- | --- |
| `pnpm poc:query-agent "问题"` | 先执行完整构建,再运行 | 首次运行,或刚改过代码 |
| `pnpm poc:query-agent:run "问题"` | 直接运行已有构建,**不构建** | 连续迭代测试 |
`poc:query-agent:run` 在构建产物不存在时会明确提示先运行 `pnpm poc:query-agent`,**不会自动构建**。
注意:`poc:query-agent` 内部走的是完整 `electron-vite build`(main + preload + renderer),
即使 POC 只需要一个 main entry。连续测试请使用 `poc:query-agent:run` 以免每次都重建整个 renderer。
## 传参
参数按原样转发给入口,可以被 `--` 分隔(`pnpm run` 惯例):
```bash
pnpm poc:query-agent -- "BOBO上个月有没有给我发过文件"
pnpm poc:query-agent:run "BOBO上个月有没有给我发过文件"
```
入口只会移除参数列表**开头**的一个独立 `--`;问题正文中的 `--` 会原样保留。
## Provider
POC 使用设置页当前默认 AI Provider、模型、Base URL 和安全存储中的 API Key。Local Query API 仍使用现有 Bearer Token;POC 输出不会打印 Token、API Key、数据库路径或内部消息 ID。
## 输出
**stdout 是 JSON**(`poc:query-agent` 会在它前面混入构建日志,`poc:query-agent:run` 只多两行 pnpm 横幅)。
需要机器解析时用 `--silent` 拿到纯 JSON:
```bash
pnpm --silent poc:query-agent:run "我和BOBO第一次聊了什么" > result.json
```
JSON 字段:
- `question`、`provider`、`model`
- `modelCallCount`、`toolCallCount`
- `modelDurationsMs`(每次模型调用耗时,含失败的那次)
- `modelDiagnostics`(每次模型调用的请求级诊断:HTTP status、content-type、是否返回 HTML、是否超时、耗时)
- `firstModelMs`、`toolTotalMs`、`finalModelMs`、`totalMs`
- 每次工具调用的名称、脱敏参数、耗时、状态和结果数量
- 最终 `answer` 或错误信息
**stderr 是人类可读摘要**(不参与 JSON 解析):
```text
[Timing]
Model #1 1315 ms
TM Tools(1) 623 ms
Model #2 1598 ms
------------------------
Total 3545 ms
Model total 2913 ms (82.2%)
TM tool total 623 ms (17.6%)
[Provider]
provider DeepSeek
model DeepSeek Chat
host api.deepseek.com
model calls 2
tool calls 1
attempt #1 elapsedMs=1298 status=200 contentType=application/json
attempt #2 elapsedMs=1571 status=200 contentType=application/json
```
`elapsedMs` 是 TTFB(收到响应头),`modelDurationsMs` 是整次调用(含读 body);502 时两者接近,
说明等待发生在上游网关,不是本地读 body 慢。诊断只记录 host,不记录完整 URL 或任何凭据。非 2xx 响应会先记录 status / content-type / elapsedMs,再返回安全错误(例如“模型服务返回了网页而不是 JSON(HTTP 502 Bad Gateway)”),不会把 HTML 正文丢给 JSON 解析器。
## 约束
工具调用最多 5 次,只允许 `query_messages`、`search_messages`、`message_context`、`conversation_overview`。未配置 AI Provider、Local Query API 未启动或当前 Provider 协议不支持 tools 时,POC 会直接返回错误,不会回退到另一套模型配置。
+2 -1
View File
@@ -41,7 +41,8 @@
"predev": "node -e \"require('electron')\"", "predev": "node -e \"require('electron')\"",
"dev": "node scripts/ensure-env.cjs && node scripts/build-wechat-connector.cjs && electron-vite dev", "dev": "node scripts/ensure-env.cjs && node scripts/build-wechat-connector.cjs && electron-vite dev",
"dev:update": "cross-env TRACEMEMO_UPDATE_SIMULATION=true pnpm dev", "dev:update": "cross-env TRACEMEMO_UPDATE_SIMULATION=true pnpm dev",
"poc:query-agent": "electron-vite build && electron out/main/queryAgentPoc.js", "poc:query-agent": "electron-vite build && node scripts/run-query-agent-poc.cjs",
"poc:query-agent:run": "node scripts/run-query-agent-poc.cjs",
"test:wechat-connector": "go -C services/wechat-connector test ./... && go -C services/wechat-connector vet ./...", "test:wechat-connector": "go -C services/wechat-connector test ./... && go -C services/wechat-connector vet ./...",
"test:unit": "vitest run --config vitest.unit.config.ts", "test:unit": "vitest run --config vitest.unit.config.ts",
"test:component": "vitest run --config vitest.component.config.ts", "test:component": "vitest run --config vitest.component.config.ts",
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env node
/**
* Query Agent POC 快速运行入口:直接执行已构建的 out/main/queryAgentPoc.js,不做任何构建。
*
* 与 `pnpm poc:query-agent` 的分工:
* - poc:query-agent : 先 electron-vite build,再运行(代码改动后使用)
* - poc:query-agent:run : 只运行现有构建产物(连续测试使用)
*
* 若构建产物不存在,给出明确提示;不会偷偷触发 full build,否则 fast-run 失去意义。
*
* 可选环境变量:
* TRACEMEMO_POC_ELECTRON 指定 Electron 可执行文件(默认取 node_modules 中的 electron)
*/
const fs = require('fs')
const path = require('path')
const { spawnSync } = require('child_process')
const repoRoot = path.resolve(__dirname, '..')
const pocEntry = path.join(repoRoot, 'out', 'main', 'queryAgentPoc.js')
if (!fs.existsSync(pocEntry)) {
process.stderr.write(
[
'',
'[poc] POC build 不存在,请先运行:',
' pnpm poc:query-agent "你的问题"',
'',
` 预期构建产物:${path.relative(repoRoot, pocEntry)}`,
' (本入口有意不自动构建,以免失去快速运行的意义)',
''
].join('\n')
)
process.exit(1)
}
// 该入口的目标就是启动 Electron 主进程,因此必须清掉会让 Electron 退化成纯 Node 的标记。
// 部分 IDE 集成终端会注入 ELECTRON_RUN_AS_NODE=1。
const env = { ...process.env }
delete env.ELECTRON_RUN_AS_NODE
// 在普通 Node 中 require('electron') 返回可执行文件路径。
const electronBinary = env.TRACEMEMO_POC_ELECTRON || require('electron')
const result = spawnSync(electronBinary, [pocEntry, ...process.argv.slice(2)], {
stdio: 'inherit',
cwd: repoRoot,
env
})
if (result.error) {
process.stderr.write(`[poc] 启动 Electron 失败:${result.error.message}\n`)
process.exit(1)
}
process.exit(typeof result.status === 'number' ? result.status : 1)
+13
View File
@@ -0,0 +1,13 @@
/**
* Query Agent POC 的 CLI 参数解析。
*
* 单独抽成纯函数,便于单测;避免 entry 文件的副作用(app bootstrap / app.whenReady)影响测试。
*/
export function parsePocQuestion(argv: readonly string[]): string {
const args = [...argv]
// npm / pnpm 在复合 script(`build && electron ...`)里会把 `--` 一并追加到命令末尾,
// 于是 separator 会落到 POC 的 argv 里,污染问题正文。
// 只移除开头的这一个独立 separator;问题正文中间的合法 `--` 必须保留。
if (args[0] === '--') args.shift()
return args.join(' ').trim()
}
+8 -2
View File
@@ -3,9 +3,11 @@ import { app } from 'electron'
import { apiTokenStore } from './api-token-store' import { apiTokenStore } from './api-token-store'
import { AIProviderService } from './services/ai-provider-service' import { AIProviderService } from './services/ai-provider-service'
import { QueryAgentPocService, type QueryAgentToolResult } from './services/query-agent-poc-service' import { QueryAgentPocService, type QueryAgentToolResult } from './services/query-agent-poc-service'
import { parsePocQuestion } from './query-agent-poc-cli'
import { formatProviderDiagnostics, formatTiming } from './query-agent-poc-report'
const baseUrl = (process.env.TRACEMEMO_QUERY_API_BASE || 'http://127.0.0.1:6131/api/v1').replace(/\/+$/, '') const baseUrl = (process.env.TRACEMEMO_QUERY_API_BASE || 'http://127.0.0.1:6131/api/v1').replace(/\/+$/, '')
const question = process.argv.slice(2).join(' ').trim() const question = parsePocQuestion(process.argv.slice(2))
async function callQueryApi(name: string, input: Record<string, unknown>): Promise<QueryAgentToolResult> { async function callQueryApi(name: string, input: Record<string, unknown>): Promise<QueryAgentToolResult> {
const paths: Record<string, string> = { const paths: Record<string, string> = {
@@ -26,9 +28,13 @@ async function callQueryApi(name: string, input: Record<string, unknown>): Promi
async function main(): Promise<void> { async function main(): Promise<void> {
await app.whenReady() await app.whenReady()
const service = new QueryAgentPocService(new AIProviderService(), callQueryApi) const provider = new AIProviderService()
const service = new QueryAgentPocService(provider, callQueryApi)
const result = await service.run(question) const result = await service.run(question)
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`) process.stdout.write(`${JSON.stringify(result, null, 2)}\n`)
// 诊断摘要写 stderr,保持 stdout 仍是纯 JSON,方便管道与脚本消费。
process.stderr.write(formatTiming(result))
process.stderr.write(formatProviderDiagnostics(result, provider.getRuntimeEndpointHost()))
app.quit() app.quit()
} }
+78
View File
@@ -0,0 +1,78 @@
import type { QueryAgentPocResult } from './services/query-agent-poc-service'
const WIDTH = 24
function row(label: string, value: string): string {
return `${label.padEnd(14)}${value.padStart(10)}`
}
/**
* 人类可读的耗时摘要。
*
* 只做展示,不改变 JSON 输出契约。需要能正确处理:
* - 0 次模型调用(配置缺失等提前返回)
* - 1 次模型调用 + 0 次工具
* - 2 次模型调用 + N 次工具(此时可准确还原"模型 → 工具 → 模型"顺序)
* - 3 次以上(无法逐段归属工具耗时时,明确标注为聚合)
* - 首次模型调用失败 / 末尾模型调用失败
*/
export function formatTiming(result: QueryAgentPocResult): string {
const durations = result.modelDurationsMs || []
const toolTotalMs = result.toolTotalMs || 0
const toolCount = result.toolCallCount || 0
const lines: string[] = ['', '[Timing]']
if (durations.length === 0) {
lines.push('(未发生模型调用)')
} else if (durations.length <= 2) {
lines.push(row('Model #1', `${durations[0]} ms`))
if (toolCount > 0) lines.push(row(`TM Tools(${toolCount})`, `${toolTotalMs} ms`))
if (durations[1] !== undefined) lines.push(row('Model #2', `${durations[1]} ms`))
} else {
durations.forEach((duration, index) => lines.push(row(`Model #${index + 1}`, `${duration} ms`)))
lines.push(row(`TM Tools(${toolCount})`, `${toolTotalMs} ms`))
lines.push('(3 次以上模型调用:工具耗时按总计展示)')
}
const modelTotalMs = durations.reduce((sum, value) => sum + value, 0)
const totalMs = result.totalMs || 0
const share = (value: number): string =>
totalMs > 0 ? `${((value / totalMs) * 100).toFixed(1)}%` : 'n/a'
lines.push('-'.repeat(WIDTH))
lines.push(row('Total', `${totalMs} ms`))
lines.push('')
lines.push(row('Model total', `${modelTotalMs} ms (${share(modelTotalMs)})`))
lines.push(row('TM tool total', `${toolTotalMs} ms (${share(toolTotalMs)})`))
if (result.error) lines.push(`error: ${result.error}`)
return `${lines.join('\n')}\n`
}
/**
* 请求级诊断。
* 只输出 host 与状态字段;绝不输出 API key / Authorization / 完整 URL / 响应正文。
*/
export function formatProviderDiagnostics(result: QueryAgentPocResult, host?: string): string {
const lines = [
'',
'[Provider]',
`provider ${result.provider}`,
`model ${result.model}`,
`host ${host || '(unknown)'}`,
`model calls ${result.modelCallCount}`,
`tool calls ${result.toolCallCount}`
]
for (const entry of result.modelDiagnostics || []) {
const parts = [`elapsedMs=${entry.elapsedMs}`]
if (entry.status !== undefined) parts.push(`status=${entry.status}`)
if (entry.contentType) parts.push(`contentType=${entry.contentType}`)
if (entry.timedOut) parts.push('timedOut=true')
if (entry.htmlInsteadOfJson) parts.push('htmlInsteadOfJson=true')
if (entry.errorCode) parts.push(`code=${entry.errorCode}`)
if (entry.errorType) parts.push(`type=${entry.errorType}`)
if (entry.status !== undefined && [502, 503, 504].includes(entry.status)) parts.push('upstreamGateway=true')
lines.push(`attempt #${entry.index} ${parts.join(' ')}`)
if (entry.error) lines.push(` error: ${entry.error}`)
}
return `${lines.join('\n')}\n`
}
+105 -15
View File
@@ -86,10 +86,16 @@ export class AIProviderRequestError extends Error {
readonly code?: string readonly code?: string
readonly type?: string readonly type?: string
readonly responseBody?: unknown readonly responseBody?: unknown
/** 诊断字段:上游响应 content-type(例如 502 返回的 text/html) */
readonly contentType?: string
/** 诊断字段:本次请求从发出到响应处理结束的耗时 */
readonly elapsedMs?: number
/** 诊断字段:上游返回网页而不是 JSON */
readonly htmlInsteadOfJson?: boolean
constructor( constructor(
message: string, message: string,
details: { status?: number; code?: unknown; type?: unknown; responseBody?: unknown } = {} details: { status?: number; code?: unknown; type?: unknown; responseBody?: unknown; contentType?: string; elapsedMs?: number; htmlInsteadOfJson?: boolean } = {}
) { ) {
super(message) super(message)
this.name = 'AIProviderRequestError' this.name = 'AIProviderRequestError'
@@ -97,6 +103,9 @@ export class AIProviderRequestError extends Error {
this.code = typeof details.code === 'string' ? details.code : undefined this.code = typeof details.code === 'string' ? details.code : undefined
this.type = typeof details.type === 'string' ? details.type : undefined this.type = typeof details.type === 'string' ? details.type : undefined
this.responseBody = details.responseBody this.responseBody = details.responseBody
this.contentType = details.contentType
this.elapsedMs = details.elapsedMs
this.htmlInsteadOfJson = details.htmlInsteadOfJson
} }
} }
@@ -136,6 +145,19 @@ export class AIProviderService {
} }
} }
/**
* 诊断用:当前默认 Provider 的 endpoint host。
* 只返回 hostname(不含路径与 query),避免把凭据或敏感 query 带进日志/终端。
*/
getRuntimeEndpointHost(): string | undefined {
try {
const resolved = this.resolveProvider(undefined)
return endpointHost(resolved.provider.baseUrl)
} catch {
return undefined
}
}
getVisionRuntimeConfig(): AIVisionRuntimeConfig { getVisionRuntimeConfig(): AIVisionRuntimeConfig {
const result = this.list() const result = this.list()
const defaultProvider = result.providers.find((item) => item.id === result.defaultProviderId) const defaultProvider = result.providers.find((item) => item.id === result.defaultProviderId)
@@ -324,6 +346,13 @@ export class AIProviderService {
toolCalls?: AIChatToolCall[] toolCalls?: AIChatToolCall[]
usage?: { input?: number; output?: number; total?: number; estimated?: boolean } usage?: { input?: number; output?: number; total?: number; estimated?: boolean }
error?: string error?: string
errorStatus?: number
errorCode?: string
errorType?: string
errorContentType?: string
elapsedMs?: number
timedOut?: boolean
htmlInsteadOfJson?: boolean
}> { }> {
try { try {
if (options?.apiKey) throw new Error('Tool Calling 不支持 legacy provider 配置') if (options?.apiKey) throw new Error('Tool Calling 不支持 legacy provider 配置')
@@ -342,7 +371,22 @@ export class AIProviderService {
return { success: true, ...result } return { success: true, ...result }
} catch (error) { } catch (error) {
if (signal?.aborted) throw error if (signal?.aborted) throw error
return { success: false, error: safeAIError(error) } const timedOut = error instanceof DOMException && error.name === 'TimeoutError'
return {
success: false,
error: safeAIError(error),
timedOut,
htmlInsteadOfJson: error instanceof AIProviderRequestError ? Boolean(error.htmlInsteadOfJson) : false,
...(error instanceof AIProviderRequestError
? {
...(error.status !== undefined ? { errorStatus: error.status } : {}),
...(error.code ? { errorCode: error.code } : {}),
...(error.type ? { errorType: error.type } : {}),
...(error.contentType ? { errorContentType: error.contentType } : {}),
...(error.elapsedMs !== undefined ? { elapsedMs: error.elapsedMs } : {})
}
: {})
}
} }
} }
@@ -598,6 +642,15 @@ function deepSeekProvider(baseUrl?: string, model?: string): AIProviderSummary {
} }
} }
/** 诊断用:只取 endpoint 的 hostname,不保留路径与 query。 */
function endpointHost(baseUrl: string): string | undefined {
try {
return new URL(baseUrl).hostname || undefined
} catch {
return undefined
}
}
function stripRuntimeFields( function stripRuntimeFields(
provider: AIProviderSummary provider: AIProviderSummary
): Omit<AIProviderSummary, 'hasApiKey' | 'isDefault'> { ): Omit<AIProviderSummary, 'hasApiKey' | 'isDefault'> {
@@ -912,7 +965,7 @@ async function requestOpenAICompatibleWithTools(
messages: Array<Record<string, unknown>>, messages: Array<Record<string, unknown>>,
tools: AIChatToolDefinition[], tools: AIChatToolDefinition[],
signal?: AbortSignal signal?: AbortSignal
): Promise<{ data: string; toolCalls: AIChatToolCall[]; usage?: AIRequestResult['usage'] }> { ): Promise<{ data: string; toolCalls: AIChatToolCall[]; usage?: AIRequestResult['usage']; elapsedMs?: number }> {
const endpoint = provider.baseUrl.endsWith('/chat/completions') const endpoint = provider.baseUrl.endsWith('/chat/completions')
? provider.baseUrl ? provider.baseUrl
: `${provider.baseUrl.replace(/\/+$/, '')}/chat/completions` : `${provider.baseUrl.replace(/\/+$/, '')}/chat/completions`
@@ -932,16 +985,33 @@ async function requestOpenAICompatibleWithTools(
}, },
provider.advanced.timeoutMs, provider.advanced.timeoutMs,
signal, signal,
async (response) => { async (response, timing) => {
const payload = await parseJsonResponse<OpenAIResponsePayload>(response) // 先尝试解析 body:非 2xx 时上游可能返回 JSON 错误体,也可能是 HTML 错误页。
if (!response.ok) { // 无论哪种,都要把 status / content-type / elapsed 带进错误,避免丢失上下文。
throw new AIProviderRequestError(payload.error?.message || `AI 请求失败 (${response.status})`, { let payload: OpenAIResponsePayload | undefined
status: response.status, let parseError: unknown
code: payload.error?.code, try {
type: payload.error?.type, payload = await parseJsonResponse<OpenAIResponsePayload>(response)
responseBody: payload.error } catch (error) {
}) parseError = error
} }
if (!response.ok) {
throw new AIProviderRequestError(
payload?.error?.message || (parseError ? safeAIError(parseError) : `AI 请求失败 (${response.status})`),
{
status: timing.status,
contentType: timing.contentType,
elapsedMs: timing.elapsedMs,
htmlInsteadOfJson: isHtmlParseError(parseError),
code: payload?.error?.code,
type: payload?.error?.type,
responseBody: payload?.error
}
)
}
// 2xx 但 body 无法解析:抛出原始解析错误(含清晰的中文提示)。
if (!payload) throw parseError ?? new Error('AI 响应格式异常')
if (parseError) throw parseError
const message = payload.choices?.[0]?.message const message = payload.choices?.[0]?.message
return { return {
data: openAIMessageText(message?.content), data: openAIMessageText(message?.content),
@@ -950,7 +1020,8 @@ async function requestOpenAICompatibleWithTools(
if (!name) return [] if (!name) return []
return [{ id: call.id || `tool-call-${index + 1}`, name, arguments: call.function?.arguments || '{}' }] return [{ id: call.id || `tool-call-${index + 1}`, name, arguments: call.function?.arguments || '{}' }]
}), }),
usage: toOpenAIUsage(payload.usage) usage: toOpenAIUsage(payload.usage),
elapsedMs: timing.elapsedMs
} }
} }
) )
@@ -1029,12 +1100,19 @@ async function requestAnthropic(
) )
} }
/** 请求级诊断:不包含任何凭据,仅用于区分「上游慢 / 上游错误 / 本地解析」。 */
interface AIRequestTiming {
elapsedMs: number
status: number
contentType: string
}
async function fetchWithTimeout<T>( async function fetchWithTimeout<T>(
url: string, url: string,
init: RequestInit, init: RequestInit,
timeoutMs: number, timeoutMs: number,
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
consume: (response: Response) => Promise<T> consume: (response: Response, timing: AIRequestTiming) => Promise<T>
): Promise<T> { ): Promise<T> {
const controller = new AbortController() const controller = new AbortController()
let timedOut = false let timedOut = false
@@ -1049,9 +1127,16 @@ async function fetchWithTimeout<T>(
}, },
Math.max(1_000, timeoutMs || 120_000) Math.max(1_000, timeoutMs || 120_000)
) )
const startedAt = Date.now()
let responseReceivedAt = startedAt
try { try {
const response = await fetch(url, { ...init, signal: controller.signal }) const response = await fetch(url, { ...init, signal: controller.signal })
return await consume(response) responseReceivedAt = Date.now()
return await consume(response, {
elapsedMs: responseReceivedAt - startedAt,
status: response.status,
contentType: response.headers.get('content-type') || ''
})
} catch (error) { } catch (error) {
if (signal?.aborted) throw new DOMException('AI request cancelled', 'AbortError') if (signal?.aborted) throw new DOMException('AI request cancelled', 'AbortError')
if (timedOut) throw new DOMException('AI request timed out', 'TimeoutError') if (timedOut) throw new DOMException('AI request timed out', 'TimeoutError')
@@ -1295,6 +1380,11 @@ function safeAIError(error: unknown): string {
return message.replace(/sk-[a-z0-9_-]+/gi, '***').slice(0, 300) return message.replace(/sk-[a-z0-9_-]+/gi, '***').slice(0, 300)
} }
/** 判断解析失败是否因为上游返回了 HTML(例如网关 502 错误页)。 */
function isHtmlParseError(error: unknown): boolean {
return error instanceof Error && error.message.includes('网页而不是 JSON')
}
function aiProviderErrorDetails(error: unknown): { function aiProviderErrorDetails(error: unknown): {
errorCode?: string errorCode?: string
errorStatus?: number errorStatus?: number
+231 -21
View File
@@ -3,6 +3,14 @@ import { LOCAL_QUERY_TOOL_DEFINITIONS } from '../../shared/local-query-api'
const MAX_TOOL_CALLS = 5 const MAX_TOOL_CALLS = 5
const FORBIDDEN_INPUT_KEYS = new Set(['apiKey', 'authorization', 'token', 'databasePath', 'sql', 'wxid', 'md5']) const FORBIDDEN_INPUT_KEYS = new Set(['apiKey', 'authorization', 'token', 'databasePath', 'sql', 'wxid', 'md5'])
// 每个工具在“首次执行但结果为 0”之后允许的额外重试次数上限。
const ZERO_RESULT_RETRY_LIMIT = 1
// absolute 时间契约:LLM 只能给带时区的 ISO-8601 字符串,Host 负责换算成 Local Query API 的 epoch seconds。
const ISO_ABSOLUTE_PATTERN = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})(?:\.\d{1,3})?(Z|[+-]\d{2}:\d{2})$/
// sanity 窗口:聊天记录不可能早于 2000 年,也不允许查询明显属于未来的区间。
const ABSOLUTE_MIN_MS = Date.UTC(2000, 0, 1)
const ABSOLUTE_MAX_FUTURE_MS = 366 * 24 * 60 * 60 * 1000
const ISO_ABSOLUTE_HINT = '带时区偏移的 ISO-8601,例如 2026-08-01T00:00:00+08:00 或 2026-07-31T16:00:00Z'
interface ToolSchema { interface ToolSchema {
type?: string type?: string
@@ -33,7 +41,21 @@ export interface QueryAgentProvider {
chatWithTools( chatWithTools(
messages: Array<Record<string, unknown>>, messages: Array<Record<string, unknown>>,
tools: AIChatToolDefinition[] tools: AIChatToolDefinition[]
): Promise<{ success: boolean; data?: string; toolCalls?: AIChatToolCall[]; usage?: { input?: number; output?: number; total?: number; estimated?: boolean }; error?: string }> ): Promise<{
success: boolean
data?: string
toolCalls?: AIChatToolCall[]
usage?: { input?: number; output?: number; total?: number; estimated?: boolean }
error?: string
/** 以下为诊断字段(additive,不参与业务语义) */
elapsedMs?: number
errorStatus?: number
errorCode?: string
errorType?: string
errorContentType?: string
timedOut?: boolean
htmlInsteadOfJson?: boolean
}>
} }
export interface QueryAgentToolResult { export interface QueryAgentToolResult {
@@ -50,6 +72,18 @@ export interface QueryAgentTraceItem {
evidenceCount?: number evidenceCount?: number
} }
export interface QueryAgentModelCallDiagnostic {
index: number
elapsedMs: number
status?: number
contentType?: string
timedOut?: boolean
htmlInsteadOfJson?: boolean
errorCode?: string
errorType?: string
error?: string
}
export interface QueryAgentPocResult { export interface QueryAgentPocResult {
question: string question: string
provider: string provider: string
@@ -60,6 +94,10 @@ export interface QueryAgentPocResult {
toolTotalMs: number toolTotalMs: number
finalModelMs?: number finalModelMs?: number
totalMs: number totalMs: number
/** 每次模型调用的耗时(含失败的那次),按调用顺序;additive 诊断字段 */
modelDurationsMs: number[]
/** 每次模型调用的请求级诊断;additive 诊断字段 */
modelDiagnostics: QueryAgentModelCallDiagnostic[]
traces: QueryAgentTraceItem[] traces: QueryAgentTraceItem[]
answer?: string answer?: string
error?: string error?: string
@@ -74,11 +112,12 @@ const SYSTEM_PROMPT = `你是 TraceMemo 的本地聊天查询助手,只能使
规划原则: 规划原则:
- 先判断问题需要哪种证据,再调用最少的 Tool。每次收到 Tool Result 后都判断“当前 Evidence 是否已经足以给出有边界的回答”;足够就立即回答,不为追求绝对完整继续调查。 - 先判断问题需要哪种证据,再调用最少的 Tool。每次收到 Tool Result 后都判断“当前 Evidence 是否已经足以给出有边界的回答”;足够就立即回答,不为追求绝对完整继续调查。
- query_messages 是精确事实查询,适用于能用联系人、时间、方向、消息类型、顺序等结构条件表达的问题。earliest/latest 等时间边界也是结构条件,必须使用 order 与 limit 精确查询,不能使用抽样 overview。结果已经回答问题时,不要追加 conversation_overview。 - query_messages 是精确事实查询,适用于能用联系人、时间、方向、消息类型、顺序等结构条件表达的问题。earliest/latest 等时间边界也是结构条件,必须使用 order 与 limit 精确查询,不能使用抽样 overview。结果已经回答问题时,不要追加 conversation_overview。若返回 0 条且你判断是时间范围或结构条件不合适,允许再查一次并合理扩大或更换条件,但必须与上一次实质不同。
- search_messages 是语义 Evidence 检索,适用于结构条件无法确定答案的问题。第一次尽量在一个调用中给出高质量 query 和最多 4 个 variants;调用前确认 variants 数组长度不超过 4。一次有效 search 后不得再次 search,已有相关 Evidence 就直接判断。 - 需要绝对时间范围时,startTime/endTime 必须使用带时区偏移的 ISO-8601 字符串(例如 2026-08-01T00:00:00+08:00 或 2026-07-31T16:00:00Z)。不要传 epoch 数字,也不要传没有时区的裸本地时间。
- conversation_overview 只用于真正需要理解一个时间范围内整体聊了什么、主要话题或整体互动的 broad summary。它返回 temporal coverage sample,不代表完整聊天,也不是语义检索不足时的默认 fallback。 - search_messages 是关键词检索,适用于结构条件无法确定答案的问题。queries 的每一项都是一次独立的字面检索:一项只放一个简短关键词,不要把多个近义词或整句话塞进同一项。首次最多 4 项。检索到 Evidence 后直接判断;只有本次完全没有 Evidence 时,才允许再检索一次,且每一项都必须与上一次实质不同。
- conversation_overview 只用于真正需要理解一个时间范围内整体聊了什么、主要话题或整体互动的 broad summary。它返回 temporal coverage sample,不代表完整聊天,也不是检索不足时的默认 fallback。
- message_context 只用于已经找到一条有价值 Evidence、但单条内容缺少前后语境而无法判断真实含义的情况。不要把它当作默认确认步骤;上下文足够后立即回答。 - message_context 只用于已经找到一条有价值 Evidence、但单条内容缺少前后语境而无法判断真实含义的情况。不要把它当作默认确认步骤;上下文足够后立即回答。
- 普通聊天查询不是 exhaustive investigation。经过合理的 search 或可选 context 仍不足以形成强结论时,直接说明证据范围和不确定性,不要循环调用 search、overview、context。 - 普通聊天查询不是 exhaustive investigation。经过合理的检索或可选 context 仍不足以形成强结论时,直接说明证据范围和不确定性,不要循环调用 search、overview、context。
事实边界:不得编造未返回的消息、猜测联系人、修改 resolvedTimeRange,或把 partial/unknown 当作 complete。coverage complete 且结果为 0 时,可以说明当前可读取的完整范围没有找到;coverage partial/unknown 且结果为 0 时,必须说明无法确认绝对不存在。不要把 sampled Evidence 当作完整聊天,也不要把 source message count 和 selected evidence count 混为一谈。 事实边界:不得编造未返回的消息、猜测联系人、修改 resolvedTimeRange,或把 partial/unknown 当作 complete。coverage complete 且结果为 0 时,可以说明当前可读取的完整范围没有找到;coverage partial/unknown 且结果为 0 时,必须说明无法确认绝对不存在。不要把 sampled Evidence 当作完整聊天,也不要把 source message count 和 selected evidence count 混为一谈。
缺少必要信息时用自然语言澄清;超出工具能力时说明不能可靠完成,并给出当前工具可以执行的替代方向。` 缺少必要信息时用自然语言澄清;超出工具能力时说明不能可靠完成,并给出当前工具可以执行的替代方向。`
@@ -187,17 +226,147 @@ function validateSchema(value: unknown, schema: ToolSchema, field = '$'): ToolAr
return undefined return undefined
} }
export function validateToolArguments(name: string, value: unknown): { input?: Record<string, unknown>; error?: ToolArgumentValidationError } { function argError(field: string, constraint: string, expected?: unknown, actual?: unknown, hint?: string): ToolArgumentValidationError {
return { status: 'invalid_tool_arguments', field, constraint, ...(expected === undefined ? {} : { expected }), ...(actual === undefined ? {} : { actual }), ...(hint ? { hint } : {}) }
}
/**
* 解析 LLM 提供的绝对时间。只接受带显式时区偏移(或 Z)的 ISO-8601;
* 无时区、epoch 数字、非法日历日一律返回 undefined,由调用方转成可修正的 invalid_tool_arguments。
*/
function parseIsoInstant(value: string): number | undefined {
const match = ISO_ABSOLUTE_PATTERN.exec(value.trim())
if (!match) return undefined
const year = Number(match[1])
const month = Number(match[2])
const day = Number(match[3])
const hour = Number(match[4])
const minute = Number(match[5])
const second = Number(match[6])
if (month < 1 || month > 12 || day < 1 || day > 31 || hour > 23 || minute > 59 || second > 59) return undefined
// 先在 UTC 语义下校验字面日期真实存在,再套用时区偏移,避免 2 月 31 日被静默进位。
const naiveMs = Date.UTC(year, month - 1, day, hour, minute, second)
const naive = new Date(naiveMs)
if (naive.getUTCFullYear() !== year || naive.getUTCMonth() !== month - 1 || naive.getUTCDate() !== day) return undefined
const offset = match[7]
if (offset === 'Z') return naiveMs
const sign = offset.startsWith('-') ? -1 : 1
const offsetHour = Number(offset.slice(1, 3))
const offsetMinute = Number(offset.slice(4, 6))
if (offsetHour > 23 || offsetMinute > 59) return undefined
return naiveMs - sign * (offsetHour * 60 + offsetMinute) * 60000
}
function canonicalizeTimeRange(timeRange: Record<string, unknown>, now: Date): { value?: Record<string, unknown>; error?: ToolArgumentValidationError } {
const startTime = timeRange.startTime
const endTime = timeRange.endTime
if (timeRange.kind !== 'absolute') {
// 非 absolute 语义下 startTime/endTime 无意义,直接丢弃,避免残留数值进入 Query API。
const { startTime: _startTime, endTime: _endTime, ...rest } = timeRange
return { value: rest }
}
if (typeof startTime !== 'string' || typeof endTime !== 'string') {
return { error: argError('timeRange.startTime', 'required', ISO_ABSOLUTE_HINT, actualType(startTime ?? endTime), ISO_ABSOLUTE_HINT) }
}
const startMs = parseIsoInstant(startTime)
if (startMs === undefined) return { error: argError('timeRange.startTime', 'format', ISO_ABSOLUTE_HINT, startTime, ISO_ABSOLUTE_HINT) }
const endMs = parseIsoInstant(endTime)
if (endMs === undefined) return { error: argError('timeRange.endTime', 'format', ISO_ABSOLUTE_HINT, endTime, ISO_ABSOLUTE_HINT) }
if (endMs < startMs) return { error: argError('timeRange.endTime', 'range_order', 'endTime 不得早于 startTime', endTime) }
const latestMs = now.getTime() + ABSOLUTE_MAX_FUTURE_MS
const sanityWindow = `2000-01-01 至 ${new Date(latestMs).toISOString()}`
if (startMs < ABSOLUTE_MIN_MS || startMs > latestMs) return { error: argError('timeRange.startTime', 'range_sanity', sanityWindow, startTime) }
if (endMs < ABSOLUTE_MIN_MS || endMs > latestMs) return { error: argError('timeRange.endTime', 'range_sanity', sanityWindow, endTime) }
// 通过全部校验后才换算成 Local Query API 使用的 epoch seconds。
return { value: { kind: 'absolute', startTime: Math.floor(startMs / 1000), endTime: Math.floor(endMs / 1000) } }
}
/**
* LLM Tool Adapter 的 canonicalization:
* - absolute 的 ISO-8601 → Local Query API 的 epoch seconds
* - search_messages 的 queries[] → Local Query API 的 query + variants
*/
function canonicalizeToolInput(name: string, input: Record<string, unknown>, now: Date): { input?: Record<string, unknown>; error?: ToolArgumentValidationError } {
const output: Record<string, unknown> = { ...input }
if (output.timeRange && typeof output.timeRange === 'object' && !Array.isArray(output.timeRange)) {
const canonical = canonicalizeTimeRange(output.timeRange as Record<string, unknown>, now)
if (canonical.error) return { error: canonical.error }
output.timeRange = canonical.value
}
if (name === 'search_messages') {
const raw = Array.isArray(output.queries) ? (output.queries as unknown[]) : []
const probes = raw.filter((value): value is string => typeof value === 'string').map((value) => value.trim()).filter(Boolean)
if (!probes.length) return { error: argError('queries', 'required', '至少一个非空检索项') }
const [first, ...rest] = probes
delete output.queries
output.query = first
if (rest.length) output.variants = rest
}
return { input: output }
}
interface ZeroResultRetryState {
searchAttempts: number
searchSignatures: string[]
queryAttempts: number
querySignatures: string[]
}
function newRetryState(): ZeroResultRetryState {
return { searchAttempts: 0, searchSignatures: [], queryAttempts: 0, querySignatures: [] }
}
function normalizedTarget(input: Record<string, unknown>): string {
const target = input.target
const query = target && typeof target === 'object' ? (target as Record<string, unknown>).query : undefined
return typeof query === 'string' ? query.trim().toLowerCase() : ''
}
/** 只覆盖“实质条件”:忽略 limit/order/excludeSystem 这类不改变检索语义的字段。 */
function retrySignature(name: string, input: Record<string, unknown>): string {
if (name === 'search_messages') {
const probes = [input.query, ...(Array.isArray(input.variants) ? input.variants : [])]
.filter((value): value is string => typeof value === 'string')
.map((value) => value.trim().toLowerCase())
return JSON.stringify({ target: normalizedTarget(input), timeRange: input.timeRange ?? null, probes: Array.from(new Set(probes)).sort() })
}
const messageTypes = Array.isArray(input.messageTypes) ? [...(input.messageTypes as string[])].sort() : []
return JSON.stringify({ target: normalizedTarget(input), timeRange: input.timeRange ?? null, direction: input.direction ?? null, messageTypes })
}
function duplicateRetry(name: string, input: Record<string, unknown>, state: ZeroResultRetryState): boolean {
const signature = retrySignature(name, input)
if (name === 'search_messages') return state.searchSignatures.includes(signature)
if (name === 'query_messages') return state.querySignatures.includes(signature)
return false
}
function recordAttempt(name: string, input: Record<string, unknown>, state: ZeroResultRetryState): void {
if (name === 'search_messages') { state.searchAttempts += 1; state.searchSignatures.push(retrySignature(name, input)) }
else if (name === 'query_messages') { state.queryAttempts += 1; state.querySignatures.push(retrySignature(name, input)) }
}
function retryNote(name: string, result: QueryAgentToolResult, state: ZeroResultRetryState): string | undefined {
if (result.constraint === 'duplicate_retry') return '本次重试的条件与上一次完全相同,已被拒绝;请改用实质不同的条件,或直接基于现有结果作答。'
if (result.status !== 'completed') return undefined
const counts = resultCount(result)
if (name === 'search_messages' && !counts.evidenceCount && state.searchAttempts <= ZERO_RESULT_RETRY_LIMIT) return '本次检索没有任何 Evidence。允许再执行一次 search_messages,但每一项都必须与上一次实质不同;完全相同的检索会被拒绝。'
if (name === 'query_messages' && counts.resultCount === 0 && state.queryAttempts <= ZERO_RESULT_RETRY_LIMIT) return '本次精确查询返回 0 条。允许再执行一次 query_messages,用于合理扩大或更换时间范围、方向或消息类型;完全相同的条件会被拒绝。'
return undefined
}
export function validateToolArguments(name: string, value: unknown, now: Date = new Date()): { input?: Record<string, unknown>; error?: ToolArgumentValidationError } {
const definition = LOCAL_QUERY_TOOL_DEFINITIONS.find((tool) => tool.name === name) const definition = LOCAL_QUERY_TOOL_DEFINITIONS.find((tool) => tool.name === name)
if (!definition) return { error: { status: 'invalid_tool_arguments', field: '$', constraint: 'tool', expected: 'supported tool', actual: name } } if (!definition) return { error: argError('$', 'tool', 'supported tool', name) }
if (!value || typeof value !== 'object' || Array.isArray(value)) { if (!value || typeof value !== 'object' || Array.isArray(value)) {
return { error: { status: 'invalid_tool_arguments', field: '$', constraint: 'type', expected: 'object', actual: actualType(value) } } return { error: argError('$', 'type', 'object', actualType(value)) }
} }
if (containsForbiddenKey(value)) { if (containsForbiddenKey(value)) {
return { error: { status: 'invalid_tool_arguments', field: '$', constraint: 'forbidden_field' } } return { error: argError('$', 'forbidden_field') }
} }
const error = validateSchema(value, definition.parameters as ToolSchema) const error = validateSchema(value, definition.parameters as ToolSchema)
return error ? { error } : { input: value as Record<string, unknown> } if (error) return { error }
return canonicalizeToolInput(name, value as Record<string, unknown>, now)
} }
function resultCount(result: QueryAgentToolResult): { resultCount?: number; evidenceCount?: number } { function resultCount(result: QueryAgentToolResult): { resultCount?: number; evidenceCount?: number } {
@@ -213,7 +382,7 @@ function messageRecordForModel(value: unknown): unknown {
return record.messageType || !record.sourceKind ? record : { ...record, messageType: record.sourceKind } return record.messageType || !record.sourceKind ? record : { ...record, messageType: record.sourceKind }
} }
function toolResultForModel(name: string, result: QueryAgentToolResult, callsUsed: number, nextTools: AIChatToolDefinition[]): QueryAgentToolResult { function toolResultForModel(name: string, result: QueryAgentToolResult, callsUsed: number, nextTools: AIChatToolDefinition[], note?: string): QueryAgentToolResult {
const visible: QueryAgentToolResult = { ...result } const visible: QueryAgentToolResult = { ...result }
if (Array.isArray(result.messages)) visible.messages = result.messages.map(messageRecordForModel) if (Array.isArray(result.messages)) visible.messages = result.messages.map(messageRecordForModel)
if (Array.isArray(result.evidence)) visible.evidence = result.evidence.map(messageRecordForModel) if (Array.isArray(result.evidence)) visible.evidence = result.evidence.map(messageRecordForModel)
@@ -225,29 +394,48 @@ function toolResultForModel(name: string, result: QueryAgentToolResult, callsUse
toolCallsUsed: callsUsed, toolCallsUsed: callsUsed,
toolCallsRemaining: Math.max(0, MAX_TOOL_CALLS - callsUsed), toolCallsRemaining: Math.max(0, MAX_TOOL_CALLS - callsUsed),
availableNextTools: nextTools.map((tool) => tool.function.name), availableNextTools: nextTools.map((tool) => tool.function.name),
instruction: nextTools.length ...(note ? { note } : {}),
instruction: [
nextTools.length
? '先判断当前 Evidence 是否足以回答;足够就立即回答,只在含义仍有明确歧义时使用当前可用 Tool。' ? '先判断当前 Evidence 是否足以回答;足够就立即回答,只在含义仍有明确歧义时使用当前可用 Tool。'
: '工具阶段已经结束。必须直接给出有边界的最终回答,不得再调用 Tool。' : '工具阶段已经结束。必须直接给出有边界的最终回答,不得再调用 Tool。',
note
].filter(Boolean).join(' ')
} }
return visible return visible
} }
function nextToolDefinitions(name: string, result: QueryAgentToolResult): AIChatToolDefinition[] { function nextToolDefinitions(name: string, result: QueryAgentToolResult, state: ZeroResultRetryState): AIChatToolDefinition[] {
// 重复重试已被拒绝,不再开放工具,避免用有限的 tool budget 反复试同一条件。
if (result.constraint === 'duplicate_retry') return []
if (result.status === 'invalid_tool_arguments') return toolDefinition(name) if (result.status === 'invalid_tool_arguments') return toolDefinition(name)
if (name === 'search_messages' && result.status === 'completed' && resultCount(result).evidenceCount) { if (result.status !== 'completed') return []
return toolDefinition('message_context') const counts = resultCount(result)
if (name === 'search_messages') {
// 有 Evidence 时保持原有高效路径:只允许补一次上下文。
if (counts.evidenceCount) return toolDefinition('message_context')
// 首次检索完全没有 Evidence:允许一次实质不同的重试,之后关闭。
return state.searchAttempts <= ZERO_RESULT_RETRY_LIMIT ? toolDefinition('search_messages') : []
}
if (name === 'query_messages') {
// 只有 0 结果才开放一次重试;有结果时保持原有 stopping。
return counts.resultCount === 0 && state.queryAttempts <= ZERO_RESULT_RETRY_LIMIT ? toolDefinition('query_messages') : []
} }
return [] return []
} }
export class QueryAgentPocService { export class QueryAgentPocService {
constructor(private readonly provider: QueryAgentProvider, private readonly executeTool: QueryAgentToolExecutor) {} constructor(
private readonly provider: QueryAgentProvider,
private readonly executeTool: QueryAgentToolExecutor,
private readonly nowProvider: () => Date = () => new Date()
) {}
async run(question: string): Promise<QueryAgentPocResult> { async run(question: string): Promise<QueryAgentPocResult> {
const trimmed = question.trim() const trimmed = question.trim()
const startedAt = Date.now() const startedAt = Date.now()
const runtime = this.provider.getRuntimeConfig() const runtime = this.provider.getRuntimeConfig()
const result: QueryAgentPocResult = { question: trimmed, provider: runtime.providerName, model: runtime.modelName || runtime.model, modelCallCount: 0, toolCallCount: 0, toolTotalMs: 0, totalMs: 0, traces: [] } const result: QueryAgentPocResult = { question: trimmed, provider: runtime.providerName, model: runtime.modelName || runtime.model, modelCallCount: 0, toolCallCount: 0, toolTotalMs: 0, totalMs: 0, traces: [], modelDurationsMs: [], modelDiagnostics: [] }
if (!trimmed) return { ...result, error: '请输入查询问题', totalMs: Date.now() - startedAt } if (!trimmed) return { ...result, error: '请输入查询问题', totalMs: Date.now() - startedAt }
if (!runtime.configured) return { ...result, error: '当前 AI Provider 尚未配置', totalMs: Date.now() - startedAt } if (!runtime.configured) return { ...result, error: '当前 AI Provider 尚未配置', totalMs: Date.now() - startedAt }
@@ -256,6 +444,8 @@ export class QueryAgentPocService {
{ role: 'user', content: trimmed } { role: 'user', content: trimmed }
] ]
let tools = toolDefinitions() let tools = toolDefinitions()
const retry = newRetryState()
const now = this.nowProvider()
let firstModelAt: number | undefined let firstModelAt: number | undefined
let finalModelDuration: number | undefined let finalModelDuration: number | undefined
while (result.toolCallCount < MAX_TOOL_CALLS) { while (result.toolCallCount < MAX_TOOL_CALLS) {
@@ -263,6 +453,19 @@ export class QueryAgentPocService {
const model = await this.provider.chatWithTools(messages, tools) const model = await this.provider.chatWithTools(messages, tools)
result.modelCallCount += 1 result.modelCallCount += 1
const modelDuration = Date.now() - modelStartedAt const modelDuration = Date.now() - modelStartedAt
// 无论成功失败都记录本次调用耗时与请求级诊断,便于区分模型慢 / 工具慢 / 上游错误。
result.modelDurationsMs.push(modelDuration)
result.modelDiagnostics.push({
index: result.modelCallCount,
elapsedMs: typeof model.elapsedMs === 'number' ? model.elapsedMs : modelDuration,
...(model.errorStatus !== undefined ? { status: model.errorStatus } : {}),
...(model.errorContentType ? { contentType: model.errorContentType } : {}),
...(model.timedOut ? { timedOut: true } : {}),
...(model.htmlInsteadOfJson ? { htmlInsteadOfJson: true } : {}),
...(model.errorCode ? { errorCode: model.errorCode } : {}),
...(model.errorType ? { errorType: model.errorType } : {}),
...(model.success ? {} : { error: model.error || '模型调用失败' })
})
if (firstModelAt === undefined) firstModelAt = Date.now() if (firstModelAt === undefined) firstModelAt = Date.now()
if (!model.success) return { ...result, error: model.error || '模型调用失败', firstModelMs: firstModelAt - startedAt, totalMs: Date.now() - startedAt } if (!model.success) return { ...result, error: model.error || '模型调用失败', firstModelMs: firstModelAt - startedAt, totalMs: Date.now() - startedAt }
const calls = model.toolCalls || [] const calls = model.toolCalls || []
@@ -292,15 +495,21 @@ export class QueryAgentPocService {
toolResult = { status: 'invalid_tool_arguments', field: '$', constraint: 'json' } toolResult = { status: 'invalid_tool_arguments', field: '$', constraint: 'json' }
} }
if (!toolResult) { if (!toolResult) {
const validated = validateToolArguments(call.name, parsed) const validated = validateToolArguments(call.name, parsed, now)
if (validated.error) { if (validated.error) {
toolResult = validated.error toolResult = validated.error
} else { } else {
traceInput = validated.input || {} traceInput = validated.input || {}
if (duplicateRetry(call.name, traceInput, retry)) {
// 明确拒绝“换关键词重搜”里的 identical retry,让模型改用实质不同的条件。
toolResult = { status: 'invalid_tool_arguments', field: '$', constraint: 'duplicate_retry', expected: '与上一次实质不同的条件', actual: '与上一次完全相同的条件' }
} else {
recordAttempt(call.name, traceInput, retry)
toolResult = await this.executeTool(call.name, traceInput) toolResult = await this.executeTool(call.name, traceInput)
} }
} }
} }
}
} catch (error) { } catch (error) {
if (!toolResult) toolResult = { status: 'invalid_request', error: error instanceof Error ? error.message : '工具调用失败' } if (!toolResult) toolResult = { status: 'invalid_request', error: error instanceof Error ? error.message : '工具调用失败' }
} }
@@ -312,8 +521,9 @@ export class QueryAgentPocService {
result.traces.push({ toolName: call.name, input: sanitizeInput(traceInput), durationMs, status: completedToolResult.status, ...counts }) result.traces.push({ toolName: call.name, input: sanitizeInput(traceInput), durationMs, status: completedToolResult.status, ...counts })
const nextTools = completedToolResult.constraint === 'tool_availability' const nextTools = completedToolResult.constraint === 'tool_availability'
? tools ? tools
: nextToolDefinitions(call.name, completedToolResult) : nextToolDefinitions(call.name, completedToolResult, retry)
messages.push({ role: 'tool', tool_call_id: call.id, name: call.name, content: JSON.stringify(toolResultForModel(call.name, completedToolResult, result.toolCallCount, nextTools)) }) const note = retryNote(call.name, completedToolResult, retry)
messages.push({ role: 'tool', tool_call_id: call.id, name: call.name, content: JSON.stringify(toolResultForModel(call.name, completedToolResult, result.toolCallCount, nextTools, note)) })
tools = nextTools tools = nextTools
} }
} }
+14 -3
View File
@@ -18,10 +18,21 @@ export interface LocalQueryToolDefinition {
parameters: Record<string, unknown> parameters: Record<string, unknown>
} }
const targetSchema = { type: 'object', properties: { query: { type: 'string', minLength: 1 } }, required: ['query'], additionalProperties: false } const targetSchema = { type: 'object', properties: { query: { type: 'string', minLength: 1 } }, required: ['query'], additionalProperties: false }
const timeRangeSchema = { type: 'object', properties: { kind: { enum: ['all', 'today', 'yesterday', 'this_week', 'last_7_days', 'this_month', 'previous_month', 'this_year', 'previous_year', 'absolute'] }, startTime: { type: 'number' }, endTime: { type: 'number' } }, required: ['kind'], additionalProperties: false } // LLM-facing 绝对时间契约:只接受带时区偏移的 ISO-8601 字符串,避免 epoch 单位歧义。
// Local Query API 内部仍使用 epoch seconds(见 QueryTimeRange),由 Host Adapter 负责转换。
const timeRangeSchema = {
type: 'object',
properties: {
kind: { enum: ['all', 'today', 'yesterday', 'this_week', 'last_7_days', 'this_month', 'previous_month', 'this_year', 'previous_year', 'absolute'] },
startTime: { type: 'string', minLength: 1, description: '仅 kind=absolute 时必填。带时区偏移的 ISO-8601 字符串,例如 2026-08-01T00:00:00+08:00 或 2026-07-31T16:00:00Z。不接受 epoch 数字,也不接受无时区的裸本地时间。' },
endTime: { type: 'string', minLength: 1, description: '仅 kind=absolute 时必填。含义与格式同 startTime,且不得早于 startTime。' }
},
required: ['kind'],
additionalProperties: false
}
export const LOCAL_QUERY_TOOL_DEFINITIONS: LocalQueryToolDefinition[] = [ export const LOCAL_QUERY_TOOL_DEFINITIONS: LocalQueryToolDefinition[] = [
{ name: 'query_messages', description: '精确读取符合联系人、时间、方向、消息类型、顺序等结构条件的消息;适合具体事实和 earliest/latest 等时间边界查询,边界查询使用 order 与 limit。', parameters: { type: 'object', required: ['target', 'timeRange'], additionalProperties: false, properties: { target: targetSchema, timeRange: timeRangeSchema, direction: { enum: ['any', 'from_target', 'to_target'] }, messageTypes: { type: 'array', items: { enum: ['text', 'image', 'voice', 'video', 'file', 'link', 'sticker', 'system', 'other'] } }, order: { enum: ['asc', 'desc'] }, limit: { type: 'integer', minimum: 1, maximum: 200 }, excludeSystem: { type: 'boolean' } } } }, { name: 'query_messages', description: '精确读取符合联系人、时间、方向、消息类型、顺序等结构条件的消息;适合具体事实和 earliest/latest 等时间边界查询,边界查询使用 order 与 limit。若结果为 0 且条件明显不合适,可再查一次并合理扩大或更换条件,但条件必须与上一次实质不同。', parameters: { type: 'object', required: ['target', 'timeRange'], additionalProperties: false, properties: { target: targetSchema, timeRange: timeRangeSchema, direction: { enum: ['any', 'from_target', 'to_target'] }, messageTypes: { type: 'array', items: { enum: ['text', 'image', 'voice', 'video', 'file', 'link', 'sticker', 'system', 'other'] } }, order: { enum: ['asc', 'desc'] }, limit: { type: 'integer', minimum: 1, maximum: 200 }, excludeSystem: { type: 'boolean' } } } },
{ name: 'search_messages', description: '在指定联系人和时间范围内按语义寻找相关 Evidence;首次调用应集中提供高质量 query 和 variants。已有相关 Evidence 时直接判断,不要重复搜索。', parameters: { type: 'object', required: ['target', 'timeRange', 'query'], additionalProperties: false, properties: { target: targetSchema, timeRange: timeRangeSchema, query: { type: 'string', minLength: 1 }, variants: { type: 'array', description: '可选的语义变体,数组元素总数不得超过 4;调用前必须自行计数。', maxItems: 4, items: { type: 'string' } }, limit: { type: 'integer', minimum: 1, maximum: 200 } } } }, { name: 'search_messages', description: '在指定联系人和时间范围内做关键词检索并返回相关 Evidence。queries 的每一项都是一次独立的字面检索:一项只放一个简短关键词,不要把多个近义词或整句话放进同一项,也不要指望一项内部被拆词理解。首次最多 4 项;只有在本次检索完全没有 Evidence 时,才允许再检索一次,且每一项都必须与上一次实质不同。', parameters: { type: 'object', required: ['target', 'timeRange', 'queries'], additionalProperties: false, properties: { target: targetSchema, timeRange: timeRangeSchema, queries: { type: 'array', description: '独立检索项列表,每项一个简短关键词,最多 4 项;每一项单独检索,不会组合成一句话理解。', minItems: 1, maxItems: 4, items: { type: 'string', minLength: 1 } }, limit: { type: 'integer', minimum: 1, maximum: 200 } } } },
{ name: 'message_context', description: '补充已找到的单条有价值 Evidence 的前后消息;仅在该 Evidence 缺少语境、无法判断含义时使用,不是默认确认步骤。', parameters: { type: 'object', required: ['messageRef'], additionalProperties: false, properties: { messageRef: { type: 'string', minLength: 1 }, before: { type: 'integer', minimum: 0, maximum: 50 }, after: { type: 'integer', minimum: 0, maximum: 50 } } } }, { name: 'message_context', description: '补充已找到的单条有价值 Evidence 的前后消息;仅在该 Evidence 缺少语境、无法判断含义时使用,不是默认确认步骤。', parameters: { type: 'object', required: ['messageRef'], additionalProperties: false, properties: { messageRef: { type: 'string', minLength: 1 }, before: { type: 'integer', minimum: 0, maximum: 50 }, after: { type: 'integer', minimum: 0, maximum: 50 } } } },
{ name: 'conversation_overview', description: '提取指定联系人和时间范围的整体聊天覆盖样本;只用于 broad summary,不是语义搜索 fallback,也不能确定 earliest/latest 等精确时间边界。', parameters: { type: 'object', required: ['target', 'timeRange'], additionalProperties: false, properties: { target: targetSchema, timeRange: timeRangeSchema } } } { name: 'conversation_overview', description: '提取指定联系人和时间范围的整体聊天覆盖样本;只用于 broad summary,不是语义搜索 fallback,也不能确定 earliest/latest 等精确时间边界。', parameters: { type: 'object', required: ['target', 'timeRange'], additionalProperties: false, properties: { target: targetSchema, timeRange: timeRangeSchema } } }
] ]
+143
View File
@@ -0,0 +1,143 @@
import { mkdtempSync, rmSync } from 'fs'
import { tmpdir } from 'os'
import { join } from 'path'
import { afterAll, describe, expect, it, vi } from 'vitest'
import type { AIProviderConfig } from '../../src/shared/ai-provider'
const root = mkdtempSync(join(tmpdir(), 'wxe-ai-provider-diagnostics-'))
vi.mock('electron', () => ({
app: { getPath: () => root },
safeStorage: {
isEncryptionAvailable: () => true,
encryptString: (value: string) => Buffer.from(value),
decryptString: (value: Buffer) => value.toString('utf8')
}
}))
import { AIProviderService } from '../../src/main/services/ai-provider-service'
function provider(baseUrl: string, timeoutMs = 120_000): AIProviderConfig {
return {
id: 'fixture-provider',
name: 'Fixture Provider',
type: 'openai-compatible' as const,
baseUrl,
auth: { type: 'none' as const },
models: [
{ id: 'fixture-model', name: 'Fixture Model', capabilities: { chat: true, vision: false, ocr: false, longContext: false } }
],
defaultModel: 'fixture-model',
advanced: { timeoutMs, extraHeaders: {} }
}
}
const TOOLS = [{ type: 'function' as const, function: { name: 'query_messages', description: 'read', parameters: { type: 'object' } } }]
describe('AI Provider 请求级诊断', () => {
afterAll(() => rmSync(root, { recursive: true, force: true }))
it('200 JSON:成功并带请求耗时,无错误诊断字段', async () => {
const service = new AIProviderService()
service.save(provider('https://diag.example.test/v1'))
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(
new Response(JSON.stringify({ choices: [{ message: { content: 'ok', tool_calls: [] } }], usage: {} }), {
status: 200,
headers: { 'content-type': 'application/json' }
})
))
try {
const result = await service.chatWithTools([{ role: 'user', content: 'q' }], TOOLS)
expect(result.success).toBe(true)
expect(typeof result.elapsedMs).toBe('number')
expect(result).not.toHaveProperty('errorStatus')
expect(result).not.toHaveProperty('htmlInsteadOfJson')
} finally {
vi.unstubAllGlobals()
}
})
it('502 text/html:保留清晰错误,并记录 status / content-type / elapsed,且不泄漏 HTML 正文', async () => {
const service = new AIProviderService()
service.save(provider('https://diag.example.test/v1'))
const html = '<!DOCTYPE html><html><head><title>502 Bad Gateway</title></head><body>relay error</body></html>'
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(
new Response(html, { status: 502, statusText: 'Bad Gateway', headers: { 'content-type': 'text/html' } })
))
try {
const result = await service.chatWithTools([{ role: 'user', content: 'q' }], TOOLS)
expect(result.success).toBe(false)
expect(result.errorStatus).toBe(502)
expect(result.errorContentType).toBe('text/html')
expect(result.htmlInsteadOfJson).toBe(true)
expect(typeof result.elapsedMs).toBe('number')
expect(result.error).toContain('网页而不是 JSON')
expect(result.error).toContain('502')
// HTML 正文不得出现在错误信息里
expect(result.error).not.toContain('relay error')
expect(result.error).not.toContain('<html')
} finally {
vi.unstubAllGlobals()
}
})
it('502 application/json:使用上游 error.message,并保留状态码', async () => {
const service = new AIProviderService()
service.save(provider('https://diag.example.test/v1'))
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(
new Response(JSON.stringify({ error: { message: 'upstream overloaded', code: 'overloaded', type: 'server_error' } }), {
status: 502,
headers: { 'content-type': 'application/json' }
})
))
try {
const result = await service.chatWithTools([{ role: 'user', content: 'q' }], TOOLS)
expect(result.success).toBe(false)
expect(result.error).toBe('upstream overloaded')
expect(result.errorStatus).toBe(502)
expect(result.errorCode).toBe('overloaded')
expect(result.errorType).toBe('server_error')
expect(result.htmlInsteadOfJson).toBe(false)
} finally {
vi.unstubAllGlobals()
}
})
it('200 但 body 不是 JSON:明确报格式异常并带状态码', async () => {
const service = new AIProviderService()
service.save(provider('https://diag.example.test/v1'))
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(
new Response('plain text body', { status: 200, headers: { 'content-type': 'text/plain' } })
))
try {
const result = await service.chatWithTools([{ role: 'user', content: 'q' }], TOOLS)
expect(result.success).toBe(false)
expect(result.error).toContain('格式异常')
expect(result.htmlInsteadOfJson).toBe(false)
} finally {
vi.unstubAllGlobals()
}
})
it('超时:标记 timedOut 且错误为 AI 请求超时', async () => {
const service = new AIProviderService()
service.save(provider('https://diag.example.test/v1', 1))
vi.stubGlobal('fetch', vi.fn((_url: string, init?: RequestInit) => new Promise<Response>((_resolve, reject) => {
init?.signal?.addEventListener('abort', () => reject(init.signal?.reason), { once: true })
})))
try {
const result = await service.chatWithTools([{ role: 'user', content: 'q' }], TOOLS)
expect(result.success).toBe(false)
expect(result.timedOut).toBe(true)
expect(result.error).toBe('AI 请求超时')
} finally {
vi.unstubAllGlobals()
}
}, 5_000)
it('只输出 host 作为 endpoint 诊断,不泄漏路径与 query', () => {
const service = new AIProviderService()
service.save(provider('https://relay.example.test/v1?token=secret-value'))
expect(service.getRuntimeEndpointHost()).toBe('relay.example.test')
})
})
+36
View File
@@ -0,0 +1,36 @@
import { describe, expect, it } from 'vitest'
import { parsePocQuestion } from '../../src/main/query-agent-poc-cli'
describe('Query Agent POC CLI 参数解析', () => {
it('移除开头的分隔符 --', () => {
expect(parsePocQuestion(['--', '我和BOBO第一次聊了什么'])).toBe('我和BOBO第一次聊了什么')
})
it('没有分隔符时保持原样', () => {
expect(parsePocQuestion(['我和BOBO第一次聊了什么'])).toBe('我和BOBO第一次聊了什么')
})
it('不删除问题正文中间的合法 --', () => {
expect(parsePocQuestion(['测试', '--', '内容'])).toBe('测试 -- 内容')
})
it('开头分隔符只移除一个,后续 -- 仍是正文', () => {
expect(parsePocQuestion(['--', '--', '内容'])).toBe('-- 内容')
expect(parsePocQuestion(['--', 'a', '--', 'b'])).toBe('a -- b')
})
it('多段参数按空格拼接,并去掉首尾空白', () => {
expect(parsePocQuestion([' ', 'BOBO', '上个月', '有没有给我发过文件 '])).toBe('BOBO 上个月 有没有给我发过文件')
})
it('只有分隔符或空参数时返回空字符串', () => {
expect(parsePocQuestion(['--'])).toBe('')
expect(parsePocQuestion([])).toBe('')
})
it('不修改传入的数组', () => {
const argv = ['--', '问题']
parsePocQuestion(argv)
expect(argv).toEqual(['--', '问题'])
})
})
+120
View File
@@ -0,0 +1,120 @@
import { describe, expect, it } from 'vitest'
import { formatProviderDiagnostics, formatTiming } from '../../src/main/query-agent-poc-report'
import type { QueryAgentPocResult } from '../../src/main/services/query-agent-poc-service'
function result(patch: Partial<QueryAgentPocResult> = {}): QueryAgentPocResult {
return {
question: 'q',
provider: 'Fixture Provider',
model: 'fixture-model',
modelCallCount: 0,
toolCallCount: 0,
toolTotalMs: 0,
totalMs: 0,
traces: [],
modelDurationsMs: [],
modelDiagnostics: [],
...patch
}
}
describe('formatTiming', () => {
it('处理 0 次模型调用', () => {
const text = formatTiming(result({ error: '当前 AI Provider 尚未配置', totalMs: 3 }))
expect(text).toContain('(未发生模型调用)')
expect(text).toContain('Total')
expect(text).toContain('error: 当前 AI Provider 尚未配置')
})
it('处理 1 次模型调用 + 0 次工具', () => {
const text = formatTiming(result({ modelCallCount: 1, modelDurationsMs: [1234], totalMs: 1234 }))
expect(text).toContain('Model #1')
expect(text).toContain('1234 ms')
expect(text).not.toContain('TM Tools')
expect(text).toContain('Model total')
expect(text).toContain('100.0%')
})
it('处理 2 次模型调用 + 1 次工具,并还原交错顺序', () => {
const text = formatTiming(
result({ modelCallCount: 2, toolCallCount: 1, modelDurationsMs: [18784, 19690], toolTotalMs: 551, totalMs: 39025 })
)
const model1 = text.indexOf('Model #1')
const tools = text.indexOf('TM Tools')
const model2 = text.indexOf('Model #2')
expect(model1).toBeGreaterThanOrEqual(0)
expect(tools).toBeGreaterThan(model1)
expect(model2).toBeGreaterThan(tools)
expect(text).toContain('38474 ms (98.6%)')
expect(text).toContain('551 ms (1.4%)')
})
it('处理首次模型调用失败', () => {
const text = formatTiming(
result({ modelCallCount: 1, modelDurationsMs: [99419], totalMs: 99419, error: '模型服务返回了网页而不是 JSON(HTTP 502 Bad Gateway)' })
)
expect(text).toContain('Model #1')
expect(text).toContain('99419 ms')
expect(text).toContain('502')
})
it('处理末尾模型调用失败(有首次耗时、无 finalModelMs)', () => {
const text = formatTiming(
result({ modelCallCount: 2, toolCallCount: 1, modelDurationsMs: [800, 1200], toolTotalMs: 300, totalMs: 2300, error: 'AI 请求超时' })
)
expect(text).toContain('Model #2')
expect(text).toContain('error: AI 请求超时')
})
it('3 次以上模型调用标注为聚合展示', () => {
const text = formatTiming(result({ modelCallCount: 3, toolCallCount: 2, modelDurationsMs: [10, 20, 30], toolTotalMs: 5, totalMs: 65 }))
expect(text).toContain('Model #3')
expect(text).toContain('工具耗时按总计展示')
})
it('totalMs 为 0 时不产生 NaN 百分比', () => {
const text = formatTiming(result({ modelCallCount: 1, modelDurationsMs: [5], totalMs: 0 }))
expect(text).not.toContain('NaN')
expect(text).toContain('n/a')
})
})
describe('formatProviderDiagnostics', () => {
it('输出 provider / model / host 与每次尝试的请求级诊断', () => {
const text = formatProviderDiagnostics(
result({
provider: 'OpenAI',
model: 'gpt-5.6-sol',
modelCallCount: 1,
modelDiagnostics: [{ index: 1, elapsedMs: 99419, status: 502, contentType: 'text/html', htmlInsteadOfJson: true, error: '模型服务返回了网页而不是 JSON(HTTP 502 Bad Gateway)' }]
}),
'relay.example.test'
)
expect(text).toContain('provider OpenAI')
expect(text).toContain('model gpt-5.6-sol')
expect(text).toContain('host relay.example.test')
expect(text).toContain('status=502')
expect(text).toContain('contentType=text/html')
expect(text).toContain('htmlInsteadOfJson=true')
expect(text).toContain('upstreamGateway=true')
expect(text).toContain('elapsedMs=99419')
})
it('标记超时', () => {
const text = formatProviderDiagnostics(
result({ modelCallCount: 1, modelDiagnostics: [{ index: 1, elapsedMs: 240000, timedOut: true, error: 'AI 请求超时' }] })
)
expect(text).toContain('timedOut=true')
expect(text).toContain('(unknown)')
})
it('不泄漏凭据:输出中不出现 token / Authorization', () => {
const text = formatProviderDiagnostics(
result({ modelDiagnostics: [{ index: 1, elapsedMs: 1, status: 200, contentType: 'application/json' }] }),
'relay.example.test'
)
expect(text).not.toMatch(/authorization/i)
expect(text).not.toMatch(/bearer/i)
expect(text).not.toMatch(/sk-/)
})
})
+246 -7
View File
@@ -32,7 +32,7 @@ describe('QueryAgentPocService', () => {
evidence: [{ messageRef: 'opaque', timestamp: 1, sender: 'BOBO', sourceKind: 'image', text: 'caption' }] evidence: [{ messageRef: 'opaque', timestamp: 1, sender: 'BOBO', sourceKind: 'image', text: 'caption' }]
})) }))
const configuredProvider = provider([ const configuredProvider = provider([
{ success: true, toolCalls: [{ id: 'call-1', name: 'search_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange: { kind: 'all' }, query: 'topic' }) }] }, { success: true, toolCalls: [{ id: 'call-1', name: 'search_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange: { kind: 'all' }, queries: ['topic'] }) }] },
{ success: true, data: '根据这条证据可以回答。' } { success: true, data: '根据这条证据可以回答。' }
]) ])
await new QueryAgentPocService(configuredProvider, execute).run('查找相关记录') await new QueryAgentPocService(configuredProvider, execute).run('查找相关记录')
@@ -88,18 +88,21 @@ describe('QueryAgentPocService', () => {
expect(configuredProvider.chatWithTools).not.toHaveBeenCalled() expect(configuredProvider.chatWithTools).not.toHaveBeenCalled()
}) })
it('accepts schema-valid variants and rejects invalid arguments before executing a tool', async () => { it('maps LLM queries[] to the Local Query API query + variants contract', async () => {
const execute = vi.fn(async () => ({ status: 'completed', returnedCount: 0 })) const execute = vi.fn(async () => ({ status: 'completed', returnedCount: 0 }))
const valid = await new QueryAgentPocService(provider([ const valid = await new QueryAgentPocService(provider([
{ success: true, toolCalls: [{ id: 'valid', name: 'search_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange: { kind: 'all' }, query: '答应', variants: ['承诺', '保证', '说好', '一定'] }) }] }, { success: true, toolCalls: [{ id: 'valid', name: 'search_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange: { kind: 'all' }, queries: ['答应', '承诺', '保证', '说好'] }) }] },
{ success: true, data: 'done' } { success: true, data: 'done' }
]), execute).run('test') ]), execute).run('test')
expect(valid.traces[0]).toMatchObject({ status: 'completed', toolName: 'search_messages' }) expect(valid.traces[0]).toMatchObject({ status: 'completed', toolName: 'search_messages' })
expect(execute).toHaveBeenCalledTimes(1) expect(execute).toHaveBeenCalledTimes(1)
// Local Query API 仍然只认 query + variants,Host 负责映射。
expect(execute.mock.calls[0][1]).toMatchObject({ query: '答应', variants: ['承诺', '保证', '说好'] })
expect(execute.mock.calls[0][1]).not.toHaveProperty('queries')
execute.mockClear() execute.mockClear()
const invalid = await new QueryAgentPocService(provider([ const invalid = await new QueryAgentPocService(provider([
{ success: true, toolCalls: [{ id: 'invalid', name: 'search_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange: { kind: 'all' }, query: '答应', variants: ['1', '2', '3', '4', '5'] }) }] }, { success: true, toolCalls: [{ id: 'invalid', name: 'search_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange: { kind: 'all' }, queries: ['1', '2', '3', '4', '5'] }) }] },
{ success: true, data: '修正后完成' } { success: true, data: '修正后完成' }
]), execute).run('test') ]), execute).run('test')
expect(invalid.traces[0]).toMatchObject({ status: 'invalid_tool_arguments' }) expect(invalid.traces[0]).toMatchObject({ status: 'invalid_tool_arguments' })
@@ -108,13 +111,249 @@ describe('QueryAgentPocService', () => {
}) })
it('enforces generic nested tool schema constraints', () => { it('enforces generic nested tool schema constraints', () => {
const base = { target: { query: 'BOBO' }, timeRange: { kind: 'all' }, query: 'x' } const base = { target: { query: 'BOBO' }, timeRange: { kind: 'all' }, queries: ['x'] }
expect(validateToolArguments('search_messages', { ...base, variants: ['1', '2', '3', '4'] }).error).toBeUndefined() expect(validateToolArguments('search_messages', { ...base, queries: ['1', '2', '3', '4'] }).error).toBeUndefined()
expect(validateToolArguments('search_messages', { ...base, variants: ['1', '2', '3', '4', '5'] }).error).toMatchObject({ field: 'variants', constraint: 'maxItems', expected: 4, actual: 5 }) expect(validateToolArguments('search_messages', { ...base, queries: ['1', '2', '3', '4', '5'] }).error).toMatchObject({ field: 'queries', constraint: 'maxItems', expected: 4, actual: 5 })
expect(validateToolArguments('search_messages', { ...base, queries: [] }).error).toMatchObject({ field: 'queries', constraint: 'minItems', expected: 1, actual: 0 })
expect(validateToolArguments('search_messages', { ...base, extra: true }).error).toMatchObject({ field: 'extra', constraint: 'additionalProperties' }) expect(validateToolArguments('search_messages', { ...base, extra: true }).error).toMatchObject({ field: 'extra', constraint: 'additionalProperties' })
// 旧的 query 字段已不再是合法属性,必须被拒绝而不是静默忽略。
expect(validateToolArguments('search_messages', { ...base, query: 'x' }).error).toMatchObject({ field: 'query', constraint: 'additionalProperties' })
// 漏掉 queries 时按 required 拒绝。
expect(validateToolArguments('search_messages', { target: { query: 'BOBO' }, timeRange: { kind: 'all' } }).error).toMatchObject({ field: 'queries', constraint: 'required' })
expect(validateToolArguments('query_messages', { target: { query: 'BOBO' }, timeRange: { kind: 'all' }, direction: 'sideways' }).error).toMatchObject({ field: 'direction', constraint: 'enum' }) expect(validateToolArguments('query_messages', { target: { query: 'BOBO' }, timeRange: { kind: 'all' }, direction: 'sideways' }).error).toMatchObject({ field: 'direction', constraint: 'enum' })
expect(validateToolArguments('query_messages', { target: { query: '' }, timeRange: { kind: 'all' } }).error).toMatchObject({ field: 'target.query', constraint: 'minLength' }) expect(validateToolArguments('query_messages', { target: { query: '' }, timeRange: { kind: 'all' } }).error).toMatchObject({ field: 'target.query', constraint: 'minLength' })
expect(validateToolArguments('query_messages', { target: { query: 'BOBO' }, timeRange: { kind: 'all' }, limit: 0 }).error).toMatchObject({ field: 'limit', constraint: 'minimum' }) expect(validateToolArguments('query_messages', { target: { query: 'BOBO' }, timeRange: { kind: 'all' }, limit: 0 }).error).toMatchObject({ field: 'limit', constraint: 'minimum' })
expect(validateToolArguments('message_context', { messageRef: 'opaque', before: 51 }).error).toMatchObject({ field: 'before', constraint: 'maximum' }) expect(validateToolArguments('message_context', { messageRef: 'opaque', before: 51 }).error).toMatchObject({ field: 'before', constraint: 'maximum' })
}) })
it('rejects a whitespace-only search probe instead of sending an empty query', () => {
const error = validateToolArguments('search_messages', { target: { query: 'BOBO' }, timeRange: { kind: 'all' }, queries: [' '] }).error
expect(error).toMatchObject({ field: 'queries', constraint: 'required' })
})
})
describe('QueryAgent absolute time contract', () => {
const now = new Date('2026-09-10T00:00:00Z')
const target = { query: 'BOBO' }
function canonical(timeRange: Record<string, unknown>) {
return validateToolArguments('query_messages', { target, timeRange }, now)
}
it('canonicalizes an ISO-8601 absolute range with offset into Local Query API epoch seconds', () => {
const result = canonical({ kind: 'absolute', startTime: '2025-08-01T00:00:00+08:00', endTime: '2025-09-01T00:00:00+08:00' })
expect(result.error).toBeUndefined()
expect(result.input?.timeRange).toEqual({
kind: 'absolute',
startTime: Math.floor(Date.parse('2025-07-31T16:00:00Z') / 1000),
endTime: Math.floor(Date.parse('2025-08-31T16:00:00Z') / 1000)
})
})
it('round-trips seconds and ISO to the same instant', () => {
const result = canonical({ kind: 'absolute', startTime: '2025-08-13T03:27:34Z', endTime: '2025-08-13T04:00:00Z' })
const startSeconds = (result.input?.timeRange as { startTime: number }).startTime
expect(startSeconds).toBe(Math.floor(1755055654000 / 1000))
expect(new Date(startSeconds * 1000).toISOString()).toBe('2025-08-13T03:27:34.000Z')
})
it('treats equivalent offsets as the same instant', () => {
const beijing = canonical({ kind: 'absolute', startTime: '2025-08-01T00:00:00+08:00', endTime: '2025-08-02T00:00:00+08:00' })
const utc = canonical({ kind: 'absolute', startTime: '2025-07-31T16:00:00Z', endTime: '2025-08-01T16:00:00Z' })
expect(beijing.input?.timeRange).toEqual(utc.input?.timeRange)
})
it('supports cross-year ranges', () => {
const result = canonical({ kind: 'absolute', startTime: '2025-12-31T20:00:00+08:00', endTime: '2026-01-01T04:00:00+08:00' })
const range = result.input?.timeRange as { startTime: number; endTime: number }
expect(result.error).toBeUndefined()
expect(range.endTime).toBeGreaterThan(range.startTime)
expect(new Date(range.startTime * 1000).toISOString()).toBe('2025-12-31T12:00:00.000Z')
expect(new Date(range.endTime * 1000).toISOString()).toBe('2025-12-31T20:00:00.000Z')
})
it('never lets millisecond epoch numbers reach the executor silently', () => {
const error = canonical({ kind: 'absolute', startTime: 1753977600000, endTime: 1756656000000 }).error
expect(error).toMatchObject({ field: 'timeRange.startTime', constraint: 'type', expected: 'string', actual: 'number' })
})
it('rejects naive local datetimes without a timezone', () => {
expect(canonical({ kind: 'absolute', startTime: '2025-08-01T00:00:00', endTime: '2025-09-01T00:00:00' }).error).toMatchObject({ field: 'timeRange.startTime', constraint: 'format' })
})
it('rejects invalid calendar days and out-of-range clock values', () => {
expect(canonical({ kind: 'absolute', startTime: '2026-02-31T00:00:00Z', endTime: '2026-03-01T00:00:00Z' }).error).toMatchObject({ field: 'timeRange.startTime', constraint: 'format' })
expect(canonical({ kind: 'absolute', startTime: '2026-03-01T25:00:00Z', endTime: '2026-03-02T00:00:00Z' }).error).toMatchObject({ field: 'timeRange.startTime', constraint: 'format' })
})
it('rejects start after end', () => {
expect(canonical({ kind: 'absolute', startTime: '2026-03-02T00:00:00Z', endTime: '2026-03-01T00:00:00Z' }).error).toMatchObject({ field: 'timeRange.endTime', constraint: 'range_order' })
})
it('rejects absurd future and past ranges instead of returning complete-zero', () => {
expect(canonical({ kind: 'absolute', startTime: '2999-01-01T00:00:00Z', endTime: '2999-02-01T00:00:00Z' }).error).toMatchObject({ field: 'timeRange.startTime', constraint: 'range_sanity' })
expect(canonical({ kind: 'absolute', startTime: '1999-01-01T00:00:00Z', endTime: '1999-02-01T00:00:00Z' }).error).toMatchObject({ field: 'timeRange.startTime', constraint: 'range_sanity' })
expect(canonical({ kind: 'absolute', startTime: '57526-01-01T00:00:00Z', endTime: '57526-02-01T00:00:00Z' }).error).toMatchObject({ field: 'timeRange.startTime', constraint: 'format' })
})
it('drops stray startTime/endTime for non-absolute kinds', () => {
expect(canonical({ kind: 'all', startTime: '2026-01-01T00:00:00Z', endTime: '2026-02-01T00:00:00Z' }).input?.timeRange).toEqual({ kind: 'all' })
expect(canonical({ kind: 'previous_month' }).input?.timeRange).toEqual({ kind: 'previous_month' })
})
it('requires both bounds for absolute ranges', () => {
expect(canonical({ kind: 'absolute', startTime: '2026-01-01T00:00:00Z' }).error).toMatchObject({ field: 'timeRange.startTime', constraint: 'required' })
})
})
describe('QueryAgent zero-result limited retry', () => {
function searchCall(id: string, queries: string[], timeRange: Record<string, unknown> = { kind: 'all' }) {
return { success: true as const, toolCalls: [{ id, name: 'search_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange, queries }) }] }
}
function queryCall(id: string, timeRange: Record<string, unknown> = { kind: 'all' }) {
return { success: true as const, toolCalls: [{ id, name: 'query_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange }) }] }
}
const emptySearch = async () => ({ status: 'completed', evidenceCount: 0, evidence: [] })
const emptyQuery = async () => ({ status: 'completed', returnedCount: 0, messages: [] })
it('keeps search_messages available once after a zero-evidence search', async () => {
const execute = vi.fn(emptySearch)
const configured = provider([searchCall('c1', ['答应']), { success: true, data: '没有找到相关证据。' }])
await new QueryAgentPocService(configured, execute).run('找承诺')
expect(vi.mocked(configured.chatWithTools).mock.calls[1]?.[1].map((tool) => tool.function.name)).toEqual(['search_messages'])
const toolMessage = vi.mocked(configured.chatWithTools).mock.calls[1]?.[0].find((message) => message.role === 'tool')
const presented = JSON.parse(String(toolMessage?.content)) as Record<string, any>
expect(presented._agent).toMatchObject({ toolCallsUsed: 1, availableNextTools: ['search_messages'] })
expect(String(presented._agent.note)).toContain('实质不同')
})
it('allows exactly one substantively different second search and then closes tools', async () => {
const execute = vi.fn(emptySearch)
const configured = provider([searchCall('c1', ['答应']), searchCall('c2', ['公积金']), { success: true, data: '仍然没有。' }])
const result = await new QueryAgentPocService(configured, execute).run('找承诺')
expect(execute).toHaveBeenCalledTimes(2)
expect(result.traces.map((trace) => trace.status)).toEqual(['completed', 'completed'])
expect(vi.mocked(configured.chatWithTools).mock.calls[2]?.[1]).toEqual([])
expect(result.error).toBeUndefined()
})
it('rejects an identical search retry instead of spending the budget again', async () => {
const execute = vi.fn(emptySearch)
const configured = provider([searchCall('c1', ['答应']), searchCall('c2', ['答应']), { success: true, data: '没有找到。' }])
const result = await new QueryAgentPocService(configured, execute).run('找承诺')
expect(execute).toHaveBeenCalledTimes(1)
expect(result.traces[1]).toMatchObject({ status: 'invalid_tool_arguments' })
expect(vi.mocked(configured.chatWithTools).mock.calls[2]?.[1]).toEqual([])
const toolMessage = vi.mocked(configured.chatWithTools).mock.calls[2]?.[0].findLast((message) => message.role === 'tool')
expect(String(toolMessage?.content)).toContain('duplicate_retry')
})
it('treats a reordered identical probe set as an identical retry', async () => {
const execute = vi.fn(emptySearch)
const configured = provider([searchCall('c1', ['答应', '承诺']), searchCall('c2', ['承诺', '答应']), { success: true, data: 'x' }])
const result = await new QueryAgentPocService(configured, execute).run('找承诺')
expect(execute).toHaveBeenCalledTimes(1)
expect(result.traces[1].status).toBe('invalid_tool_arguments')
})
it('keeps the efficient path when the first search already returned evidence', async () => {
const execute = vi.fn(async () => ({ status: 'completed', evidenceCount: 2, evidence: [{ messageRef: 'ref', text: 't' }] }))
const configured = provider([searchCall('c1', ['答应']), { success: true, data: 'ok' }])
await new QueryAgentPocService(configured, execute).run('找承诺')
expect(vi.mocked(configured.chatWithTools).mock.calls[1]?.[1].map((tool) => tool.function.name)).toEqual(['message_context'])
})
it('allows one query_messages retry after a zero-result exact query', async () => {
const execute = vi.fn()
.mockResolvedValueOnce({ status: 'completed', returnedCount: 0, messages: [] })
.mockResolvedValueOnce({ status: 'completed', returnedCount: 1, messages: [{ messageRef: 'ref' }] })
const configured = provider([queryCall('c1', { kind: 'last_7_days' }), queryCall('c2', { kind: 'all' }), { success: true, data: '找到。' }])
const result = await new QueryAgentPocService(configured, execute).run('找文件')
expect(execute).toHaveBeenCalledTimes(2)
expect(execute.mock.calls[1][1]).toMatchObject({ timeRange: { kind: 'all' } })
expect(result.traces[1].status).toBe('completed')
})
it('rejects an identical query_messages retry', async () => {
const execute = vi.fn(emptyQuery)
const configured = provider([queryCall('c1', { kind: 'previous_month' }), queryCall('c2', { kind: 'previous_month' }), { success: true, data: 'x' }])
const result = await new QueryAgentPocService(configured, execute).run('找文件')
expect(execute).toHaveBeenCalledTimes(1)
expect(result.traces[1].status).toBe('invalid_tool_arguments')
})
it('keeps the efficient stop when the exact query already returned messages', async () => {
const execute = vi.fn(async () => ({ status: 'completed', returnedCount: 3, messages: [] }))
const configured = provider([queryCall('c1'), { success: true, data: 'ok' }])
await new QueryAgentPocService(configured, execute).run('找')
expect(vi.mocked(configured.chatWithTools).mock.calls[1]?.[1]).toEqual([])
})
it('still caps total tool calls at five', async () => {
const execute = vi.fn(emptyQuery)
const responses = [
queryCall('c1', { kind: 'last_7_days' }),
queryCall('c2', { kind: 'all' }),
queryCall('c3', { kind: 'today' }),
queryCall('c4', { kind: 'yesterday' }),
queryCall('c5', { kind: 'this_year' }),
queryCall('c6', { kind: 'this_month' })
]
const result = await new QueryAgentPocService(provider(responses), execute).run('找')
expect(result.toolCallCount).toBe(5)
expect(result.error).toContain('最大工具调用次数')
})
})
describe('QueryAgent 耗时与请求级诊断记录', () => {
function queryCall(id: string) {
return { success: true as const, toolCalls: [{ id, name: 'query_messages', arguments: JSON.stringify({ target: { query: 'BOBO' }, timeRange: { kind: 'all' }, limit: 1 }) }] }
}
it('记录每次模型调用耗时,包括失败的那次(首次调用失败)', async () => {
const configured = provider([
{ success: false, error: '模型服务返回了网页而不是 JSON(HTTP 502 Bad Gateway)', errorStatus: 502, errorContentType: 'text/html', htmlInsteadOfJson: true, elapsedMs: 99419 }
])
const result = await new QueryAgentPocService(configured, vi.fn()).run('测试')
expect(result.modelCallCount).toBe(1)
expect(result.modelDurationsMs).toHaveLength(1)
expect(result.modelDiagnostics).toEqual([
expect.objectContaining({ index: 1, status: 502, contentType: 'text/html', htmlInsteadOfJson: true, elapsedMs: 99419 })
])
expect(result.error).toContain('502')
// 首次模型调用就失败,不应产生任何工具调用
expect(result.toolCallCount).toBe(0)
expect(result.finalModelMs).toBeUndefined()
})
it('2 次模型调用 + 1 次工具:耗时数组与调用次数一致', async () => {
const configured = provider([queryCall('c1'), { success: true, data: '完成' }])
const result = await new QueryAgentPocService(configured, vi.fn(async () => ({ status: 'completed', returnedCount: 1 }))).run('测试')
expect(result.modelCallCount).toBe(2)
expect(result.modelDurationsMs).toHaveLength(2)
expect(result.modelDurationsMs.every((value) => typeof value === 'number' && value >= 0)).toBe(true)
expect(result.modelDiagnostics.map((entry) => entry.index)).toEqual([1, 2])
expect(result.toolCallCount).toBe(1)
})
it('1 次模型调用 + 0 次工具:无工具诊断噪声', async () => {
const configured = provider([{ success: true, data: '直接回答' }])
const result = await new QueryAgentPocService(configured, vi.fn()).run('测试')
expect(result.modelCallCount).toBe(1)
expect(result.toolCallCount).toBe(0)
expect(result.modelDurationsMs).toHaveLength(1)
expect(result.modelDiagnostics[0]).not.toHaveProperty('error')
})
it('末尾模型调用失败时两条记录都在,且第二条带错误', async () => {
const configured = provider([
queryCall('c1'),
{ success: false, error: 'AI 请求超时', timedOut: true }
])
const result = await new QueryAgentPocService(configured, vi.fn(async () => ({ status: 'completed', returnedCount: 1 }))).run('测试')
expect(result.modelDurationsMs).toHaveLength(2)
expect(result.modelDiagnostics[1]).toMatchObject({ index: 2, timedOut: true })
expect(result.error).toBe('AI 请求超时')
expect(result.answer).toBeUndefined()
})
}) })
+63
View File
@@ -0,0 +1,63 @@
import { copyFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs'
import { tmpdir } from 'os'
import { join } from 'path'
import { spawnSync } from 'child_process'
import { afterAll, describe, expect, it } from 'vitest'
const repoRoot = process.cwd()
const sandboxes: string[] = []
/** 搭一个最小"仓库":只放 runner 脚本,可选放假的构建产物。 */
function sandbox(withBuild: boolean): string {
const dir = mkdtempSync(join(tmpdir(), 'wxe-poc-run-'))
sandboxes.push(dir)
mkdirSync(join(dir, 'scripts'), { recursive: true })
copyFileSync(join(repoRoot, 'scripts', 'run-query-agent-poc.cjs'), join(dir, 'scripts', 'run-query-agent-poc.cjs'))
if (withBuild) {
mkdirSync(join(dir, 'out', 'main'), { recursive: true })
writeFileSync(
join(dir, 'out', 'main', 'queryAgentPoc.js'),
'process.stdout.write(JSON.stringify({ ranEntry: true, argv: process.argv.slice(2) }) + "\\n")\n'
)
}
return dir
}
function run(dir: string, args: string[], env: Record<string, string> = {}) {
return spawnSync(process.execPath, [join(dir, 'scripts', 'run-query-agent-poc.cjs'), ...args], {
encoding: 'utf8',
cwd: dir,
env: { ...process.env, ...env }
})
}
describe('poc:query-agent:run 快速入口', () => {
afterAll(() => {
for (const dir of sandboxes) rmSync(dir, { recursive: true, force: true })
})
it('构建缺失时给出明确提示并退出码 1,不触发构建', () => {
const result = run(sandbox(false), ['问题'])
expect(result.status).toBe(1)
expect(result.stderr).toContain('POC build 不存在')
expect(result.stderr).toContain('pnpm poc:query-agent')
expect(result.stderr).toContain('不自动构建')
expect(result.stdout).toBe('')
})
it('构建存在时直接执行,且原样转发参数(分隔符交给 entry 处理)', () => {
const dir = sandbox(true)
const result = run(dir, ['--', '我和BOBO第一次聊了什么'], { TRACEMEMO_POC_ELECTRON: process.execPath })
expect(result.status).toBe(0)
const payload = JSON.parse(result.stdout.trim()) as { ranEntry: boolean; argv: string[] }
expect(payload.ranEntry).toBe(true)
// runner 不修改参数:leading `--` 保留,由 POC entry 决定是否移除。
expect(payload.argv).toEqual(['--', '我和BOBO第一次聊了什么'])
})
it('构建存在但启动器失败时返回非 0 退出码并给出提示', () => {
const result = run(sandbox(true), ['问题'], { TRACEMEMO_POC_ELECTRON: '/nonexistent/electron-binary' })
expect(result.status).toBe(1)
expect(result.stderr).toContain('启动 Electron 失败')
})
})