diff --git a/README.md b/README.md index f1d8013..7aefc2d 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ - 生成改动文档同步计划:按变更文件自动提示需要同步的 README、API、AGENTS、docs、Obsidian、skill 和 ktWorkflow 入口。 - 生成多仓库提交/推送计划:按仓库分组、建议提交信息、列出提交和推送前检查。 - 生成远程只读健康检查和数据库同步安全向导:覆盖飞牛 NAS 服务探测、GTID、`.kt-workspace/db-sync` 转储、备份库和行数校验。 +- 生成或执行部署观测:把 Jenkins build、commit、K8s Deployment、Pod、`/health/runtime` 和任务 smoke 汇总成 `.kt-workspace/test-artifacts/deploy-observation` 下的运行态证据。 - 生成专项组件工作流:KtTable、BlogArgon、AdminAuth、QQBot、FF14Plugin、NapCatLogin、SystemLog、Knife4jSwagger、FnosK8s 的防踩坑清单和验证点。 - 生成验证进程清理计划:按项目路径和端口给出 PowerShell 检查命令,不直接杀进程。 - 清理历史产物:统一治理 `.kt-workspace` 下的测试/验证产物,按目录最近修改时间只保留最近 3 轮,模板目录永久保留;CLI 默认 dry-run,真实清理必须显式传 `--execute`。 @@ -40,6 +41,7 @@ pnpm run obsidian-sync pnpm run workstream-closeout -- --title "发布闭环" --verification "Jenkins SUCCESS" --doc-sync "无需文档更新" --cleanup "cleanup-history dry-run deleted=0" --cleanup-final-deleted 0 --review "global-review findings=0" --superpowers-review "Superpowers reviewer completed; no Critical/Important findings" --problem "无新卡点" --solution "无新增稳定解法" pnpm run cleanup-history -- --dry-run pnpm run cleanup-history -- --execute +pnpm run deploy-observation -- --project api --job KT-Template/KT-Template-API/main --namespace kt-prod --deployment kt-template-online-api --container api --health-url http://127.0.0.1:48085/health/runtime --smoke "curl -fsS --max-time 8 http://127.0.0.1:48085/health/runtime" pnpm run admin-login -- --url http://127.0.0.1:5999/#/auth/login ``` @@ -93,6 +95,7 @@ pnpm run admin-login -- --url http://127.0.0.1:5999/#/auth/login | `kt_business_test_plan` | 生成固化业务链路测试计划,包含 QQBot SSE、FFLogs 和系统日志 | | `kt_blocker_resolution` | 生成或写入卡点固化记录,提醒停止原样重试 | | `kt_remote_health_check` | 生成或执行远程只读健康检查命令 | +| `kt_deploy_observation` | 生成或执行 API 发布后的部署观测,汇总 Jenkins、K8s、Pod、`/health/runtime` 和任务 smoke 证据 | | `kt_db_sync_plan` | 生成数据库同步安全向导 | | `kt_component_workflow` | 输出专项组件/链路防踩坑工作流,包含 FnosK8s | | `kt_obsidian_context` | 读取 Obsidian 索引上下文,支持按 `module` / `query` 返回模块页、文档矩阵、Canvas 摘要、代码锚点和相关规则文档 | @@ -107,6 +110,7 @@ pnpm run admin-login -- --url http://127.0.0.1:5999/#/auth/login | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pnpm run admin-login` | 使用可见 Edge 打开 Admin 登录页,填写账号密码,拖动滑块,保存登录态和截图。默认账号来自初始化数据 `admin/123456`,生产或个人账号用 `KT_ADMIN_USERNAME` / `KT_ADMIN_PASSWORD` 或 CLI 参数覆盖。 | | `pnpm run global-review` | 对 KT 全部子仓库做只读全局 CodeReview 扫描,默认仅对变更文件做内容深扫,输出 JSON 复审报告,并校验 `TASKS.md` 最近记录只保留范围、关键词、验证字段、NapCat `latest` 漂移、QQBot 重型服务单测导入风险、OneBot/QQ 登录态边界、NapCat 密码登录验证码 wait 阶段、日志窗口锚点、验证码状态早于日志 URL 的短轮询、验证码 pending 被状态轮询误清理、API Pod SSH 日志读取超时及清理前 pending 回归,以及普通处理中状态误吃旧验证码日志回归;明显测试占位 token 不报凭据风险,确认误报时优先升级 `src/tools/review.ts`,不要把误报沉积到上下文。 | +| `pnpm run deploy-observation` | 默认 dry-run 输出只读 NAS 观测命令;传 `--execute` 时执行 Jenkins/K8s/health/smoke 观测并写入 `.kt-workspace/test-artifacts/deploy-observation`。 | | `pnpm run obsidian-context` | 输出 Obsidian 索引上下文;可传 `--module Admin`、`--query QQBot`、`--max-documents 10`。 | | `pnpm run obsidian-validate` | 校验 KT Obsidian vault 结构和链接;默认 warning 不让脚本失败,需要严格模式时传 `--fail-on-warnings`。 | | `pnpm run obsidian-sync` | 审计 Obsidian 工作流入口是否连通,并联动执行 validate。 | @@ -145,12 +149,14 @@ pnpm run admin-login -- --url http://127.0.0.1:5999/#/auth/login - 要收尾时调用 `kt_finish_task`,默认只生成计划和 review;需要执行验证时显式传 `runValidation=true`。 - 大方向结束前调用 `kt_workstream_closeout` 或 `pnpm run workstream-closeout`;必须分别传入 `--review` 的 KT 全局审查证据和 `--superpowers-review` 的 Superpowers code review 证据;如果识别到测试流、卡点、误报、清理、部署观测或命令模板,要先升级对应 ktWorkflow 规则再报告完成。 - 文件改动完成并验证后调用 `kt_global_code_review`,或运行 `pnpm run global-review`;它只读扫描,不删除文件、不提交代码。`findings` 先判定真实风险或工具误报,真实风险修业务代码,误报修 `mcp/ktWorkflow/src/tools/review.ts` 并复跑。 +- QQBot NapCat 登录 review 会检查验证码 pending、Docker 日志窗口、API Pod 读取超时,以及 `needNewDevice/jumpUrl` 是否被保留为 `deviceVerifyUrl` pending 状态。 - 代码或配置改动后调用 `kt_change_doc_sync`,把需要同步的 README/API/AGENTS/docs/Obsidian/skill/ktWorkflow 入口补齐;无需同步时把原因写进收尾证据。 - 报告非平凡任务完成前调用 `kt_workflow_loop_audit`,确认测试证据、文档同步、历史清理最终 `deleted=0`、问题固化、ktWorkflow 升级、KT 全局 review 和 Superpowers code review 都过门;清理证据必须显式传入,不能依赖缺省值。 - 要提交或推送时先调用 `kt_commit_plan` / `kt_push_plan`,按仓库分组确认范围。 - 要测真实业务链路时调用 `kt_business_test_plan`,选择 `admin-login`、`qqbot-auto-reply`、`qqbot-login-sse`、`fflogs-command`、`system-log-visualization` 等 flow。 - 遇到同一命令或同一远程步骤重复卡住时调用 `kt_blocker_resolution`;第二次仍失败时先写入 `TASKS.md` 或补成脚本,再继续。 - 远程服务排查先调用 `kt_remote_health_check`,默认只生成只读命令;需要执行时显式传 `execute=true`。 +- API 推送触发 Jenkins/K8s 后先运行 `pnpm run deploy-observation -- --project api --job KT-Template/KT-Template-API/main --namespace kt-prod --deployment kt-template-online-api --container api --health-url http://127.0.0.1:48085/health/runtime --smoke "curl -fsS --max-time 8 http://127.0.0.1:48085/health/runtime"` 查看 dry-run 命令;确认需要线上只读观测时再加 `--execute`。Deployment/Pod 成功只算发布证据,功能完成还必须看 smoke 输出。 - 数据库同步前调用 `kt_db_sync_plan`,先确认源库、目标库、`.kt-workspace/db-sync` 转储目录、带时间戳的备份库和校验点。 - 改 KtTable、BlogArgon、AdminAuth、QQBot、FF14Plugin、NapCatLogin、SystemLog、Knife4jSwagger、FnosK8s 前调用 `kt_component_workflow`,先看专项禁区。 - 不确定任务边界时调用 `kt_guardrails`,先拿到“能做什么、不能做什么、怎么验证”。 diff --git a/package.json b/package.json index b98e187..f34e1b1 100644 --- a/package.json +++ b/package.json @@ -7,6 +7,7 @@ "scripts": { "admin-login": "node --import ./node_modules/tsx/dist/loader.mjs scripts/admin-login-smoke.ts", "cleanup-history": "node --import ./node_modules/tsx/dist/loader.mjs src/server.ts --cleanup-history", + "deploy-observation": "node --import ./node_modules/tsx/dist/loader.mjs src/server.ts --deploy-observation", "global-review": "node --import ./node_modules/tsx/dist/loader.mjs src/server.ts --global-review", "obsidian-context": "node --import ./node_modules/tsx/dist/loader.mjs src/server.ts --obsidian-context", "obsidian-sync": "node --import ./node_modules/tsx/dist/loader.mjs src/server.ts --obsidian-sync", diff --git a/src/core/cli.ts b/src/core/cli.ts index 9f505cd..3f53ba6 100644 --- a/src/core/cli.ts +++ b/src/core/cli.ts @@ -1,4 +1,5 @@ import type { + DeployObservationInput, GlobalCodeReviewInput, ObsidianContextInput, ObsidianSyncInput, @@ -90,6 +91,38 @@ export function parseWorkstreamCloseoutCliArgs(argv: string[]): WorkstreamCloseo }; } +export function parseDeployObservationCliArgs(argv: string[]): DeployObservationInput { + const sshPort = Number(readOption(argv, ['--ssh-port', '--sshPort'])); + + return { + artifactRoot: readOption(argv, ['--artifact-root', '--artifactRoot']), + buildNumber: readOption(argv, ['--build', '--build-number', '--buildNumber']), + container: readOption(argv, ['--container']), + deployment: readOption(argv, ['--deployment']), + execute: argv.includes('--execute'), + expectedCommit: readOption(argv, [ + '--commit', + '--expected-commit', + '--expectedCommit', + ]), + healthUrl: readOption(argv, ['--health-url', '--healthUrl']), + imageTag: readOption(argv, ['--image-tag', '--imageTag']), + jenkinsHome: readOption(argv, ['--jenkins-home', '--jenkinsHome']), + jobName: readOption(argv, ['--job', '--job-name', '--jobName']), + kubeconfigPath: readOption(argv, [ + '--kubeconfig', + '--kubeconfig-path', + '--kubeconfigPath', + ]), + namespace: readOption(argv, ['--namespace']), + project: readOption(argv, ['--project']), + selector: readOption(argv, ['--selector']), + smoke: readOption(argv, ['--smoke']), + sshPort: Number.isFinite(sshPort) ? sshPort : undefined, + sshTarget: readOption(argv, ['--ssh', '--ssh-target', '--sshTarget']), + }; +} + export function parseObsidianCliArgs(argv: string[]): ObsidianValidateInput { return { checkLayout: !argv.includes('--no-layout'), diff --git a/src/core/constants.ts b/src/core/constants.ts index 630da20..3474c3c 100644 --- a/src/core/constants.ts +++ b/src/core/constants.ts @@ -71,6 +71,7 @@ export const registeredToolNames = [ 'kt_business_test_plan', 'kt_blocker_resolution', 'kt_remote_health_check', + 'kt_deploy_observation', 'kt_db_sync_plan', 'kt_component_workflow', 'kt_obsidian_context', diff --git a/src/registerTools.ts b/src/registerTools.ts index aa9e2a9..574055d 100644 --- a/src/registerTools.ts +++ b/src/registerTools.ts @@ -16,6 +16,7 @@ import { buildBusinessTestPlan, createApiTestPlan, createPageTestCase } from './ import { buildVerificationPlan } from './tools/verification.js'; import { buildWorkstreamCloseout } from './tools/closeout.js'; import { buildChangeDocSync } from './tools/docSync.js'; +import { buildDeployObservation } from './tools/deployObservation.js'; import { buildCommitPlan, buildComponentWorkflow, buildDbSyncPlan, buildFinishTask, buildPushPlan, buildRemoteHealthCheck } from './tools/workflow.js'; import { workstreamReusablePatternValues } from './core/constants.js'; export function response(data: unknown): McpTextResponse { @@ -400,6 +401,37 @@ export function registerTools(server: McpServer): void { async (input) => response(await buildRemoteHealthCheck(input)), ); + server.registerTool( + 'kt_deploy_observation', + { + description: + '生成或执行 API 发布后的只读部署观测:Jenkins、K8s Deployment、Pod、/health/runtime 和任务 smoke,并输出运行态证据。', + inputSchema: { + artifactRoot: z.string().optional(), + buildNumber: z.string().optional(), + container: z.string().default('api'), + deployment: z.string().default('kt-template-online-api'), + execute: z.boolean().default(false), + expectedCommit: z.string().optional(), + healthUrl: z.string().default('http://127.0.0.1:48085/health/runtime'), + imageTag: z.string().optional(), + jenkinsHome: z.string().default('/vol1/docker/jenkins/jenkins_home'), + jobName: z.string().default('KT-Template/KT-Template-API/main'), + kubeconfigPath: z + .string() + .default('/vol1/docker/kt-k8s/kubeconfig/kt-nas.jenkins.yaml'), + namespace: z.string().default('kt-prod'), + project: z.string().default('api'), + selector: z.string().default('app=kt-template-online-api'), + smoke: z.string().optional(), + sshPort: z.number().int().min(1).max(65_535).optional(), + sshTarget: z.string().default('nas'), + }, + title: 'KT Deploy Observation', + }, + async (input) => response(await buildDeployObservation(input)), + ); + server.registerTool( 'kt_db_sync_plan', { diff --git a/src/selfTest.ts b/src/selfTest.ts index 8761cf9..aea08f8 100644 --- a/src/selfTest.ts +++ b/src/selfTest.ts @@ -2,6 +2,7 @@ import { mkdirSync, writeFileSync } from "node:fs"; import path from "node:path"; import { + parseDeployObservationCliArgs, parseGlobalReviewCliArgs, parseWorkstreamCloseoutCliArgs, } from "./core/cli.js"; @@ -35,6 +36,11 @@ import { import { buildBusinessTestPlan } from "./tools/testing.js"; import { buildVerificationPlan } from "./tools/verification.js"; import { buildWorkstreamCloseout } from "./tools/closeout.js"; +import { + buildDeployObservation, + buildDeployObservationSectionMap, + normalizeDeployObservationEvidence, +} from "./tools/deployObservation.js"; import { buildCommitPlan, buildComponentWorkflow, @@ -167,6 +173,9 @@ export async function runSelfTest(): Promise { " return this.failCaptchaLogin(session, container, status.loginError);", " }", "}", + "if (captchaResult?.needNewDevice && captchaResult.jumpUrl) {", + " return this.keepSessionPending(session, '请在 NapCat WebUI 继续完成', true);", + "}", "private async tryPasswordRelogin(session, container) {", " await this.containerService.ensureRuntimeLoginEnv(container, { loginPassword: 'x' });", " session.lastRestartedAt = Date.now();", @@ -185,6 +194,7 @@ export async function runSelfTest(): Promise { "qqbot-napcat-captcha-status-before-log-url", "qqbot-napcat-captcha-status-clears-pending", "qqbot-napcat-captcha-log-window-late", + "qqbot-napcat-new-device-drops-pending", ]) { if (!qqbotCaptchaFlowFindings.some((item) => item.category === category)) { throw new Error(`QQBot captcha flow self-check failed: ${category}`); @@ -208,6 +218,13 @@ export async function runSelfTest(): Promise { " }", " }", "}", + "if (captchaResult?.needNewDevice && captchaResult.jumpUrl) {", + " return this.keepDeviceVerifyPending(session, captchaResult.jumpUrl, '验证码已通过');", + "}", + "private keepDeviceVerifyPending(session, deviceVerifyUrl) {", + " session.deviceVerifyUrl = deviceVerifyUrl;", + " session.captchaUrl = undefined;", + "}", "private async detectPasswordCaptchaUrl(container, sinceMs, allowTailFallback = true) {", " await this.containerService.detectRuntimeCaptchaUrl(container, sinceMs);", " await this.containerService.detectRuntimeCaptchaUrl(container);", @@ -335,6 +352,461 @@ export async function runSelfTest(): Promise { throw new Error("workstream closeout review CLI parser self-check failed"); } + const deployObservationInput = parseDeployObservationCliArgs([ + "node", + "server", + "--deploy-observation", + "--project", + "api", + "--job", + "kt-template-online-api", + "--build", + "132", + "--commit", + "abc1234", + "--image-tag", + "abc1234", + "--namespace", + "kt-prod", + "--deployment", + "kt-template-online-api", + "--container", + "api", + "--health-url", + "http://127.0.0.1:48085/health/runtime", + "--smoke", + "curl -fsS --max-time 8 http://127.0.0.1:48085/health/runtime", + ]); + if ( + deployObservationInput.project !== "api" || + deployObservationInput.jobName !== "kt-template-online-api" || + deployObservationInput.buildNumber !== "132" || + deployObservationInput.expectedCommit !== "abc1234" || + deployObservationInput.imageTag !== "abc1234" || + deployObservationInput.namespace !== "kt-prod" || + deployObservationInput.deployment !== "kt-template-online-api" || + deployObservationInput.container !== "api" || + deployObservationInput.healthUrl !== + "http://127.0.0.1:48085/health/runtime" || + deployObservationInput.smoke !== + "curl -fsS --max-time 8 http://127.0.0.1:48085/health/runtime" || + deployObservationInput.execute !== false + ) { + throw new Error("deploy observation CLI parser self-check failed"); + } + + const buildDeployObservationTestSections = (overrides: { + desiredReplicas?: number; + healthStatus?: string; + omitSmoke?: boolean; + readyReplicas?: number; + smokeExitCode?: number; + containerName?: string; + podContainerName?: string; + updatedReplicas?: number; + } = {}) => + buildDeployObservationSectionMap([ + "__KT_SECTION:jenkins_meta__", + "buildNumber=132", + "result=SUCCESS", + "__KT_SECTION:jenkins_log_tail__", + "Checking out Revision abc1234", + "Finished: SUCCESS", + "__KT_SECTION:deployment_json__", + JSON.stringify({ + metadata: { generation: 12 }, + spec: { + replicas: overrides.desiredReplicas ?? 1, + template: { + spec: { + containers: [ + { + image: + "k3d-kt-registry.localhost:5000/kt-template-online-api:abc1234", + name: overrides.containerName ?? "api", + }, + ], + }, + }, + }, + status: { + observedGeneration: 12, + readyReplicas: overrides.readyReplicas ?? 1, + updatedReplicas: overrides.updatedReplicas ?? 1, + }, + }), + "__KT_SECTION:pods_json__", + JSON.stringify({ + items: [ + { + metadata: { + name: "kt-template-online-api-abc", + }, + spec: { + containers: [ + { + image: + "k3d-kt-registry.localhost:5000/kt-template-online-api:abc1234", + name: overrides.podContainerName ?? "api", + }, + ], + }, + status: { + containerStatuses: [ + { + image: + "k3d-kt-registry.localhost:5000/kt-template-online-api:abc1234", + name: overrides.podContainerName ?? "api", + ready: true, + restartCount: 0, + }, + ], + phase: "Running", + startTime: "2026-06-14T00:00:00Z", + }, + }, + ], + }), + "__KT_SECTION:health_json__", + JSON.stringify({ + checkedAt: "2026-06-14T00:00:01.000Z", + checks: [ + { + critical: true, + message: "NestJS process answered runtime health request", + name: "process", + status: "live", + }, + ], + service: "kt-template-online-api", + status: overrides.healthStatus ?? "ready", + }), + "__KT_SECTION:smoke_text__", + overrides.omitSmoke + ? "no task smoke command provided" + : '{"service":"kt-template-online-api","status":"ready"}', + "__KT_SECTION:smoke_status__", + overrides.omitSmoke ? "exitCode=" : `exitCode=${overrides.smokeExitCode ?? 0}`, + ].join("\n")); + const deployObservationSections = buildDeployObservationTestSections(); + const deployObservationEvidence = normalizeDeployObservationEvidence({ + checkedAt: new Date("2026-06-14T00:00:02.000Z"), + input: deployObservationInput, + sectionMap: deployObservationSections, + }); + const deployObservationSmoke = deployObservationEvidence.details.smoke as { + exitCode?: number | null; + success?: boolean | null; + }; + if ( + deployObservationEvidence.status !== "passed" || + deployObservationEvidence.details.deployment.image !== + "k3d-kt-registry.localhost:5000/kt-template-online-api:abc1234" || + deployObservationEvidence.details.deployment.observedGeneration !== 12 || + deployObservationEvidence.details.pod.name !== + "kt-template-online-api-abc" || + deployObservationEvidence.details.pod.restartCount !== 0 || + deployObservationEvidence.details.runtimeHealth.status !== "ready" || + deployObservationSmoke.exitCode !== 0 || + deployObservationSmoke.success !== true || + !deployObservationEvidence.assertions.every((item) => item.passed) + ) { + throw new Error("deploy observation evidence normalization self-check failed"); + } + + const deployObservationDegradedHealthEvidence = + normalizeDeployObservationEvidence({ + checkedAt: new Date("2026-06-14T00:00:02.000Z"), + input: deployObservationInput, + sectionMap: buildDeployObservationTestSections({ + healthStatus: "degraded", + }), + }); + if ( + deployObservationDegradedHealthEvidence.status !== "passed" || + !deployObservationDegradedHealthEvidence.assertions.every( + (item) => item.passed, + ) + ) { + throw new Error("deploy observation degraded health self-check failed"); + } + + const deployObservationFailedSmokeEvidence = + normalizeDeployObservationEvidence({ + checkedAt: new Date("2026-06-14T00:00:02.000Z"), + input: deployObservationInput, + sectionMap: buildDeployObservationTestSections({ + smokeExitCode: 7, + }), + }); + const deployObservationFailedSmoke = + deployObservationFailedSmokeEvidence.details.smoke as { + exitCode?: number | null; + success?: boolean | null; + }; + if ( + deployObservationFailedSmokeEvidence.status === "passed" || + deployObservationFailedSmoke.exitCode !== 7 || + deployObservationFailedSmoke.success !== false + ) { + throw new Error("deploy observation failed smoke self-check failed"); + } + + const deployObservationReplicaLagEvidence = + normalizeDeployObservationEvidence({ + checkedAt: new Date("2026-06-14T00:00:02.000Z"), + input: deployObservationInput, + sectionMap: buildDeployObservationTestSections({ + desiredReplicas: 2, + readyReplicas: 1, + updatedReplicas: 2, + }), + }); + if (deployObservationReplicaLagEvidence.status === "passed") { + throw new Error("deploy observation replica lag self-check failed"); + } + + const deployObservationDesiredReplicaLagEvidence = + normalizeDeployObservationEvidence({ + checkedAt: new Date("2026-06-14T00:00:02.000Z"), + input: deployObservationInput, + sectionMap: buildDeployObservationTestSections({ + desiredReplicas: 2, + readyReplicas: 1, + updatedReplicas: 1, + }), + }); + if (deployObservationDesiredReplicaLagEvidence.status === "passed") { + throw new Error("deploy observation desired replica lag self-check failed"); + } + + const deployObservationWrongContainerEvidence = + normalizeDeployObservationEvidence({ + checkedAt: new Date("2026-06-14T00:00:02.000Z"), + input: deployObservationInput, + sectionMap: buildDeployObservationTestSections({ + containerName: "sidecar", + podContainerName: "sidecar", + }), + }); + if (deployObservationWrongContainerEvidence.status === "passed") { + throw new Error("deploy observation wrong container self-check failed"); + } + + const deployObservationNoSmokeEvidence = normalizeDeployObservationEvidence({ + checkedAt: new Date("2026-06-14T00:00:02.000Z"), + input: { ...deployObservationInput, smoke: undefined }, + sectionMap: buildDeployObservationTestSections({ + omitSmoke: true, + }), + }); + const deployObservationNoSmoke = deployObservationNoSmokeEvidence.details + .smoke as { + exitCode?: number | null; + success?: boolean | null; + }; + if ( + deployObservationNoSmokeEvidence.status === "passed" || + deployObservationNoSmoke.exitCode !== null || + deployObservationNoSmoke.success !== false + ) { + throw new Error("deploy observation no-smoke self-check failed"); + } + + const deployObservationDryRun = await buildDeployObservation({ + deployment: "kt-template-online-api", + execute: false, + imageTag: "abc1234", + jobName: "kt-template-online-api", + namespace: "kt-prod", + project: "api", + smoke: "curl -fsS --max-time 8 http://127.0.0.1:48085/health/runtime", + }); + const deployObservationCommandText = deployObservationDryRun.commands + .map((item) => item.command) + .join("\n"); + if ( + deployObservationDryRun.execute !== false || + !deployObservationCommandText.includes("tr -d '\\015' | bash -s") || + !deployObservationCommandText.includes("kubectl") || + !deployObservationCommandText.includes( + "http://127.0.0.1:48085/health/runtime", + ) || + !deployObservationCommandText.includes("curl -fsS --max-time 8") || + deployObservationDryRun.evidence + ) { + throw new Error("deploy observation dry-run self-check failed"); + } + for (const expected of [ + "KUBECONFIG_ACTIVE", + "docker port k3d-kt-nas-serverlb 6443/tcp", + "/tmp/kt-deploy-observation-kubeconfig-$$.yaml", + "127.0.0.1:${K3D_API_PORT}", + 'rm -f "$KUBECONFIG_TEMP"', + 'kubectl --kubeconfig "$KUBECONFIG_ACTIVE"', + ]) { + if (!deployObservationCommandText.includes(expected)) { + throw new Error(`deploy observation kubeconfig self-check failed: ${expected}`); + } + } + if ( + deployObservationCommandText.includes('sed -E "s#.*:([0-9]+)$#') || + !deployObservationCommandText.includes("rev | cut -d: -f1 | rev") + ) { + throw new Error("deploy observation docker port parser self-check failed"); + } + + const deployObservationDefaultDryRun = await buildDeployObservation({ + deployment: "kt-template-online-api", + execute: false, + namespace: "kt-prod", + project: "api", + }); + const deployObservationDefaultCommandText = + deployObservationDefaultDryRun.commands.map((item) => item.command).join("\n"); + if ( + !deployObservationDefaultCommandText.includes( + "/vol1/docker/jenkins/jenkins_home", + ) || + !deployObservationDefaultCommandText.includes( + "JOB_NAME='KT-Template/KT-Template-API/main'", + ) + ) { + throw new Error("deploy observation Jenkins defaults self-check failed"); + } + for (const expected of [ + 'JOB_DIR="$JENKINS_HOME/jobs/$JOB_NAME/builds"', + 'jobs/$JOB_FOLDER/jobs/$JOB_PIPELINE/branches/$JOB_BRANCH/builds', + 'find "$JENKINS_HOME/jobs"', + ]) { + if (!deployObservationDefaultCommandText.includes(expected)) { + throw new Error( + `deploy observation Jenkins path self-check failed: ${expected}`, + ); + } + } + + const deployObservationExplicitJenkinsDryRun = await buildDeployObservation({ + execute: false, + jenkinsHome: "/custom/jenkins", + jobName: "custom-job", + }); + const deployObservationExplicitJenkinsCommandText = + deployObservationExplicitJenkinsDryRun.commands + .map((item) => item.command) + .join("\n"); + if ( + !deployObservationExplicitJenkinsCommandText.includes( + "JENKINS_HOME='/custom/jenkins'", + ) || + !deployObservationExplicitJenkinsCommandText.includes("JOB_NAME='custom-job'") + ) { + throw new Error("deploy observation explicit Jenkins input self-check failed"); + } + + const deployObservationExplicitHealthDryRun = await buildDeployObservation({ + deployment: "kt-template-online-api", + execute: false, + healthUrl: "http://127.0.0.1:30085/health/runtime", + imageTag: "abc1234", + jobName: "kt-template-online-api", + namespace: "kt-prod", + project: "api", + }); + const deployObservationExplicitHealthCommandText = + deployObservationExplicitHealthDryRun.commands + .map((item) => item.command) + .join("\n"); + if ( + !deployObservationExplicitHealthCommandText.includes( + "http://127.0.0.1:30085/health/runtime", + ) + ) { + throw new Error("deploy observation explicit health URL self-check failed"); + } + + const deployObservationUnsafeProjectDryRun = await buildDeployObservation({ + deployment: "kt-template-online-api", + execute: false, + healthUrl: "http://127.0.0.1:48085/health/runtime", + imageTag: "abc1234", + jobName: "kt-template-online-api", + namespace: "kt-prod", + project: "../escape", + smoke: "curl -fsS --max-time 8 http://127.0.0.1:48085/health/runtime", + }); + if ( + deployObservationUnsafeProjectDryRun.execute !== false || + deployObservationUnsafeProjectDryRun.artifactPath || + deployObservationUnsafeProjectDryRun.evidence || + JSON.stringify(deployObservationUnsafeProjectDryRun.commands).includes( + "../escape", + ) + ) { + throw new Error("deploy observation project dry-run safety self-check failed"); + } + + let unsupportedSmokeRejected = false; + try { + await buildDeployObservation({ + execute: false, + smoke: + "curl -fsS --max-time 8 http://127.0.0.1:48085/health/runtime; rm -rf /", + }); + } catch (error) { + unsupportedSmokeRejected = String(error).includes( + "Unsupported deploy observation smoke command", + ); + } + if (!unsupportedSmokeRejected) { + throw new Error("deploy observation unsafe smoke self-check failed"); + } + + let unsafeHealthUrlRejected = false; + try { + await buildDeployObservation({ + execute: false, + healthUrl: "http://user:pass@127.0.0.1:48085/health/runtime", + }); + } catch (error) { + unsafeHealthUrlRejected = String(error).includes( + "Unsupported deploy observation healthUrl", + ); + } + if (!unsafeHealthUrlRejected) { + throw new Error("deploy observation unsafe health URL self-check failed"); + } + + let unsafeSshTargetRejected = false; + try { + await buildDeployObservation({ + execute: true, + sshTarget: "root@unexpected-host.example", + }); + } catch (error) { + unsafeSshTargetRejected = String(error).includes( + 'Unsafe deploy observation input "sshTarget"', + ); + } + if (!unsafeSshTargetRejected) { + throw new Error("deploy observation unsafe ssh target self-check failed"); + } + + let unsafeInputRejected = false; + try { + await buildDeployObservation({ + execute: false, + jobName: "bad\njob", + }); + } catch (error) { + unsafeInputRejected = String(error).includes( + "Unsafe deploy observation input", + ); + } + if (!unsafeInputRejected) { + throw new Error("deploy observation unsafe input self-check failed"); + } + const refactorGuardrails = buildGuardrails({ project: "mcp", taskType: "refactor", diff --git a/src/server.ts b/src/server.ts index b1e6915..24a41c2 100644 --- a/src/server.ts +++ b/src/server.ts @@ -3,6 +3,7 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'; import { + parseDeployObservationCliArgs, parseGlobalReviewCliArgs, parseObsidianContextCliArgs, parseObsidianCliArgs, @@ -11,6 +12,7 @@ import { } from './core/cli.js'; import { cleanupHistoryArtifacts, parseCliCleanupArgs } from './tools/cleanup.js'; import { buildWorkstreamCloseout } from './tools/closeout.js'; +import { buildDeployObservation } from './tools/deployObservation.js'; import { readObsidianContext, syncObsidianWorkflow, validateObsidianVault } from './tools/obsidian.js'; import { buildGlobalCodeReview } from './tools/review.js'; import { registerTools } from './registerTools.js'; @@ -32,6 +34,14 @@ if (process.argv.includes('--cleanup-history')) { console.log(JSON.stringify(readObsidianContext(parseObsidianContextCliArgs(process.argv)), null, 2)); } else if (process.argv.includes('--obsidian-sync')) { console.log(JSON.stringify(syncObsidianWorkflow(parseObsidianSyncCliArgs(process.argv)), null, 2)); +} else if (process.argv.includes('--deploy-observation')) { + console.log( + JSON.stringify( + await buildDeployObservation(parseDeployObservationCliArgs(process.argv)), + null, + 2, + ), + ); } else if (process.argv.includes('--self-test')) { await runSelfTest(); } else if (process.argv.includes('--workstream-closeout')) { diff --git a/src/tools/closeout.ts b/src/tools/closeout.ts index 7f387b2..7d56da2 100644 --- a/src/tools/closeout.ts +++ b/src/tools/closeout.ts @@ -26,9 +26,9 @@ const upgradeTargets = { prompt: '把反复使用且容易写错的命令模板封装到工作流、业务测试计划或脚本。', }, 'deploy-observation': { - files: ['src/tools/workflow.ts', 'src/tools/testing.ts', 'README.md'], + files: ['src/tools/deployObservation.ts', 'src/tools/workflow.ts', 'src/tools/testing.ts', 'README.md'], label: '部署观测', - prompt: '把 Jenkins/K8s build、commit、镜像、Deployment、Pod、日志和 smoke 观测步骤写成固定计划。', + prompt: '把 Jenkins/K8s build、commit、镜像、Deployment、Pod、/health/runtime、日志和 smoke 观测步骤写成固定计划。', }, none: { files: [], diff --git a/src/tools/deployObservation.ts b/src/tools/deployObservation.ts new file mode 100644 index 0000000..5042de7 --- /dev/null +++ b/src/tools/deployObservation.ts @@ -0,0 +1,749 @@ +import { mkdirSync, writeFileSync } from 'node:fs'; +import path from 'node:path'; + +import type { + DeployObservationAssertion, + DeployObservationCommand, + DeployObservationDeploymentEvidence, + DeployObservationEvidence, + DeployObservationInput, + DeployObservationJenkinsEvidence, + DeployObservationPodEvidence, + DeployObservationResult, + DeployObservationRuntimeHealthEvidence, + DeployObservationStatus, +} from '../types.js'; +import { tryPowerShell } from '../core/exec.js'; +import { + formatDateInShanghai, + resolveInsideRoot, + workspaceRoot, +} from '../core/workspace.js'; + +const defaultArtifactRoot = '.kt-workspace/test-artifacts/deploy-observation'; +const unsafeHereStringPattern = /[\r\n]|'@|"@/; + +interface NormalizedInput + extends Required< + Pick< + DeployObservationInput, + | 'artifactRoot' + | 'container' + | 'deployment' + | 'healthUrl' + | 'jenkinsHome' + | 'jobName' + | 'kubeconfigPath' + | 'namespace' + | 'project' + | 'selector' + | 'sshTarget' + > + > { + buildNumber?: string; + execute: boolean; + expectedCommit?: string; + imageTag?: string; + smoke?: string; + sshPort?: number; +} + +interface NormalizeEvidenceInput { + checkedAt?: Date; + input: DeployObservationInput; + sectionMap: Map; +} + +function normalizeArtifactRoot(value?: string): string { + const artifactRoot = value?.trim() || defaultArtifactRoot; + assertSafeStructuredInput('artifactRoot', artifactRoot); + const base = resolveInsideRoot(defaultArtifactRoot); + const target = resolveInsideRoot(artifactRoot); + const relative = path.relative(base, target); + + if (relative.startsWith('..') || path.isAbsolute(relative)) { + throw new Error( + `Deploy observation artifacts must stay under ${defaultArtifactRoot}`, + ); + } + + return artifactRoot; +} + +function assertSafeStructuredInput(field: string, value?: string): void { + if (!value) return; + if (unsafeHereStringPattern.test(value)) { + throw new Error( + `Unsafe deploy observation input "${field}": CR/LF and PowerShell here-string terminators are not allowed.`, + ); + } +} + +function cleanStructuredInput( + field: string, + value: string | undefined, +): string | undefined { + const trimmed = value?.trim(); + assertSafeStructuredInput(field, trimmed); + return trimmed || undefined; +} + +function normalizeSshTarget(value?: string, execute = false): string { + const target = cleanStructuredInput('sshTarget', value) || 'nas'; + if (target.startsWith('-')) { + throw new Error( + 'Unsafe deploy observation input "sshTarget": SSH option injection is not allowed.', + ); + } + if (execute && target !== 'nas') { + throw new Error( + 'Unsafe deploy observation input "sshTarget": execute mode only allows the stabilized "nas" SSH target.', + ); + } + return target; +} + +function normalizeHttpUrl(field: string, value: string): string { + let parsed: URL; + try { + parsed = new URL(value); + } catch { + throw new Error( + `Unsupported deploy observation ${field}: value must be a valid http(s) URL.`, + ); + } + + if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') { + throw new Error( + `Unsupported deploy observation ${field}: URL must use http or https.`, + ); + } + if (parsed.username || parsed.password) { + throw new Error( + `Unsupported deploy observation ${field}: URL credentials are not allowed.`, + ); + } + + return parsed.toString(); +} + +function normalizeSmokeCommand(value?: string): string | undefined { + const smoke = value?.trim(); + if (!smoke) return undefined; + assertSafeStructuredInput('smoke', smoke); + + const match = smoke.match( + /^curl\s+-fsS\s+--max-time\s+([1-9]\d?)\s+(['"]?)(https?:\/\/[^\s'"`;$|&<>\\]+)\2$/, + ); + if (!match) { + throw new Error( + 'Unsupported deploy observation smoke command: only bounded curl -fsS --max-time GET smoke is allowed.', + ); + } + + const timeout = Number(match[1]); + const url = match[3]; + if (!Number.isInteger(timeout) || timeout < 1 || timeout > 30) { + throw new Error( + 'Unsupported deploy observation smoke command: --max-time must be between 1 and 30 seconds.', + ); + } + + const parsedUrl = normalizeHttpUrl('smoke command', url); + + return `curl -fsS --max-time ${timeout} ${shellSingleQuote(parsedUrl)}`; +} + +function normalizeInput(input: DeployObservationInput = {}): NormalizedInput { + const deployment = + cleanStructuredInput('deployment', input.deployment) || + 'kt-template-online-api'; + const healthUrl = + cleanStructuredInput('healthUrl', input.healthUrl) || + 'http://127.0.0.1:48085/health/runtime'; + const execute = input.execute === true; + + return { + artifactRoot: normalizeArtifactRoot(input.artifactRoot), + buildNumber: cleanStructuredInput('buildNumber', input.buildNumber), + container: cleanStructuredInput('container', input.container) || 'api', + deployment, + execute, + expectedCommit: cleanStructuredInput( + 'expectedCommit', + input.expectedCommit, + ), + healthUrl: normalizeHttpUrl('healthUrl', healthUrl), + imageTag: cleanStructuredInput('imageTag', input.imageTag), + jenkinsHome: + cleanStructuredInput('jenkinsHome', input.jenkinsHome) || + '/vol1/docker/jenkins/jenkins_home', + jobName: + cleanStructuredInput('jobName', input.jobName) || + 'KT-Template/KT-Template-API/main', + kubeconfigPath: + cleanStructuredInput('kubeconfigPath', input.kubeconfigPath) || + '/vol1/docker/kt-k8s/kubeconfig/kt-nas.jenkins.yaml', + namespace: cleanStructuredInput('namespace', input.namespace) || 'kt-prod', + project: cleanStructuredInput('project', input.project) || 'api', + selector: cleanStructuredInput('selector', input.selector) || `app=${deployment}`, + smoke: normalizeSmokeCommand(input.smoke), + sshPort: + typeof input.sshPort === 'number' && Number.isFinite(input.sshPort) + ? input.sshPort + : undefined, + sshTarget: normalizeSshTarget(input.sshTarget, execute), + }; +} + +function shellSingleQuote(value: string): string { + return `'${value.replaceAll("'", "'\"'\"'")}'`; +} + +function powerShellArgument(value: string): string { + if (/^[A-Za-z0-9._@:/-]+$/.test(value)) return value; + return `'${value.replaceAll("'", "''")}'`; +} + +function numberOrNull(value: unknown): number | null { + return typeof value === 'number' && Number.isFinite(value) ? value : null; +} + +function stringOrNull(value: unknown): string | null { + return typeof value === 'string' && value.length > 0 ? value : null; +} + +function parseJsonObject(value: string): Record | null { + try { + const parsed = JSON.parse(value); + return parsed && typeof parsed === 'object' && !Array.isArray(parsed) + ? (parsed as Record) + : null; + } catch { + return null; + } +} + +function getPath(source: unknown, keys: string[]): unknown { + return keys.reduce((current, key) => { + if (!current || typeof current !== 'object') return undefined; + return (current as Record)[key]; + }, source); +} + +function getArray(source: unknown, keys: string[]): unknown[] { + const value = getPath(source, keys); + return Array.isArray(value) ? value : []; +} + +function objectAt(value: unknown): Record | null { + return value && typeof value === 'object' && !Array.isArray(value) + ? (value as Record) + : null; +} + +function findNamedObject( + values: unknown[], + name: string, +): Record | null { + const objects = values + .map(objectAt) + .filter((item): item is Record => Boolean(item)); + return objects.find((item) => stringOrNull(item.name) === name) || null; +} + +function readKeyValueSection(value: string): Record { + return Object.fromEntries( + value + .split(/\r?\n/) + .map((line) => { + const index = line.indexOf('='); + return index >= 0 + ? [line.slice(0, index).trim(), line.slice(index + 1).trim()] + : null; + }) + .filter((item): item is [string, string] => Boolean(item?.[0])), + ); +} + +function buildRemoteScript(input: NormalizedInput): string { + const smokeBlock = input.smoke + ? [ + 'section smoke_text', + 'set +e', + `${input.smoke} 2>&1`, + 'SMOKE_EXIT=$?', + 'set -u', + 'section smoke_status', + 'printf "exitCode=%s\\n" "$SMOKE_EXIT"', + ].join('\n') + : [ + 'section smoke_text', + "echo 'no task smoke command provided'", + 'section smoke_status', + 'printf "exitCode=\\n"', + ].join('\n'); + + return [ + 'set -u', + 'section() { printf "\\n__KT_SECTION:%s__\\n" "$1"; }', + `JOB_NAME=${shellSingleQuote(input.jobName)}`, + `BUILD_NUMBER=${shellSingleQuote(input.buildNumber || '')}`, + `JENKINS_HOME=${shellSingleQuote(input.jenkinsHome)}`, + `KUBECONFIG_PATH=${shellSingleQuote(input.kubeconfigPath)}`, + `NAMESPACE=${shellSingleQuote(input.namespace)}`, + `DEPLOYMENT=${shellSingleQuote(input.deployment)}`, + `SELECTOR=${shellSingleQuote(input.selector)}`, + `HEALTH_URL=${shellSingleQuote(input.healthUrl)}`, + 'KUBECONFIG_ACTIVE="$KUBECONFIG_PATH"', + 'KUBECONFIG_TEMP=""', + 'cleanup_kubeconfig_temp() { if [ -n "$KUBECONFIG_TEMP" ]; then rm -f "$KUBECONFIG_TEMP"; fi; }', + 'trap cleanup_kubeconfig_temp EXIT', + 'if [ -f "$KUBECONFIG_PATH" ] && grep -q "https://k3d-kt-nas-serverlb:6443" "$KUBECONFIG_PATH"; then', + ' K3D_API_PORT="$(docker port k3d-kt-nas-serverlb 6443/tcp 2>/dev/null | tail -1 | rev | cut -d: -f1 | rev || true)"', + ' if [ -n "$K3D_API_PORT" ]; then', + ' KUBECONFIG_TEMP="/tmp/kt-deploy-observation-kubeconfig-$$.yaml"', + ' sed "s#https://k3d-kt-nas-serverlb:6443#https://127.0.0.1:${K3D_API_PORT}#g" "$KUBECONFIG_PATH" > "$KUBECONFIG_TEMP"', + ' KUBECONFIG_ACTIVE="$KUBECONFIG_TEMP"', + ' fi', + 'fi', + 'JOB_DIR="$JENKINS_HOME/jobs/$JOB_NAME/builds"', + 'if [ ! -d "$JOB_DIR" ]; then', + ' JOB_SLASH_COUNT="$(printf "%s" "$JOB_NAME" | awk -F/ \'{print NF-1}\')"', + ' if [ "$JOB_SLASH_COUNT" = "2" ]; then', + ' JOB_FOLDER="${JOB_NAME%%/*}"', + ' JOB_BRANCH="${JOB_NAME##*/}"', + ' JOB_PIPELINE="${JOB_NAME#*/}"', + ' JOB_PIPELINE="${JOB_PIPELINE%/*}"', + ' MULTIBRANCH_JOB_DIR="$JENKINS_HOME/jobs/$JOB_FOLDER/jobs/$JOB_PIPELINE/branches/$JOB_BRANCH/builds"', + ' if [ -d "$MULTIBRANCH_JOB_DIR" ]; then', + ' JOB_DIR="$MULTIBRANCH_JOB_DIR"', + ' else', + ' FOUND_JOB_DIR="$(find "$JENKINS_HOME/jobs" -path "*/jobs/$JOB_PIPELINE/branches/$JOB_BRANCH/builds" -type d -print -quit 2>/dev/null || true)"', + ' if [ -z "$FOUND_JOB_DIR" ]; then FOUND_JOB_DIR="$(find "$JENKINS_HOME/jobs" -path "*/jobs/$JOB_PIPELINE/branches/main/builds" -type d -print -quit 2>/dev/null || true)"; fi', + ' if [ -n "$FOUND_JOB_DIR" ]; then JOB_DIR="$FOUND_JOB_DIR"; fi', + ' fi', + ' fi', + 'fi', + 'if [ -z "$BUILD_NUMBER" ] && [ -d "$JOB_DIR" ]; then BUILD_NUMBER="$(find "$JOB_DIR" -maxdepth 1 -type d -printf "%f\\n" 2>/dev/null | grep -E "^[0-9]+$" | sort -n | tail -1 || true)"; fi', + 'BUILD_DIR="$JOB_DIR/$BUILD_NUMBER"', + 'section jenkins_meta', + 'printf "jobName=%s\\n" "$JOB_NAME"', + 'printf "buildNumber=%s\\n" "$BUILD_NUMBER"', + 'if [ -f "$BUILD_DIR/build.xml" ]; then RESULT="$(grep -Eo "[^<]*" "$BUILD_DIR/build.xml" 2>/dev/null | sed -E "s###g" | tail -1 || true)"; else RESULT=""; fi', + 'printf "result=%s\\n" "$RESULT"', + 'section jenkins_log_tail', + 'if [ -f "$BUILD_DIR/log" ]; then tail -n 160 "$BUILD_DIR/log"; else echo "jenkins log not found: $BUILD_DIR/log"; fi', + 'section deployment_json', + 'kubectl --kubeconfig "$KUBECONFIG_ACTIVE" --request-timeout=8s -n "$NAMESPACE" get deployment "$DEPLOYMENT" -o json 2>&1 || true', + 'section pods_json', + 'kubectl --kubeconfig "$KUBECONFIG_ACTIVE" --request-timeout=8s -n "$NAMESPACE" get pods -l "$SELECTOR" -o json 2>&1 || true', + 'section events_tail', + 'kubectl --kubeconfig "$KUBECONFIG_ACTIVE" --request-timeout=8s -n "$NAMESPACE" get events --sort-by=.lastTimestamp 2>&1 | tail -n 80 || true', + 'section health_json', + 'curl -fsS --max-time 8 "$HEALTH_URL" 2>&1 || true', + smokeBlock, + ].join('\n'); +} + +export function buildDeployObservationCommands( + input: DeployObservationInput = {}, +): DeployObservationCommand[] { + const normalized = normalizeInput(input); + const sshParts = [ + 'ssh', + ...(normalized.sshPort ? ['-p', String(normalized.sshPort)] : []), + powerShellArgument(normalized.sshTarget), + '"tr -d \'\\015\' | bash -s"', + ]; + const command = [ + "$script = @'", + buildRemoteScript(normalized), + "'@", + `$script | ${sshParts.join(' ')}`, + ].join('\n'); + + return [ + { + command, + name: 'nas-deploy-observation', + }, + ]; +} + +export function buildDeployObservationSectionMap( + stdout: string, +): Map { + const sections = new Map(); + let current: string | null = null; + let buffer: string[] = []; + + for (const line of stdout.split(/\r?\n/)) { + const marker = line.match(/^__KT_SECTION:([A-Za-z0-9_-]+)__$/); + if (marker) { + if (current) sections.set(current, buffer.join('\n').trim()); + current = marker[1]; + buffer = []; + } else if (current) { + buffer.push(line); + } + } + + if (current) sections.set(current, buffer.join('\n').trim()); + + return sections; +} + +function normalizeJenkinsEvidence( + input: NormalizedInput, + sectionMap: Map, +): DeployObservationJenkinsEvidence { + const meta = readKeyValueSection(sectionMap.get('jenkins_meta') || ''); + const logTail = sectionMap.get('jenkins_log_tail') || ''; + const finishedStatus = + meta.result || + logTail.match(/Finished:\s*([A-Z_]+)/i)?.[1]?.toUpperCase() || + null; + const expectedCommit = input.expectedCommit || null; + + return { + buildNumber: meta.buildNumber || input.buildNumber || null, + commitMatched: expectedCommit + ? logTail.toLowerCase().includes(expectedCommit.toLowerCase()) + : null, + expectedCommit, + finishedStatus, + jobName: input.jobName, + }; +} + +function normalizeDeploymentEvidence( + input: NormalizedInput, + deploymentJson: Record | null, +): DeployObservationDeploymentEvidence { + const container = findNamedObject( + getArray(deploymentJson, ['spec', 'template', 'spec', 'containers']), + input.container, + ); + + return { + container: input.container, + containerFound: Boolean(container), + desiredReplicas: + numberOrNull(getPath(deploymentJson, ['spec', 'replicas'])) ?? + numberOrNull(getPath(deploymentJson, ['status', 'replicas'])) ?? + 1, + generation: numberOrNull(getPath(deploymentJson, ['metadata', 'generation'])), + image: stringOrNull(container?.image), + namespace: input.namespace, + observedGeneration: numberOrNull( + getPath(deploymentJson, ['status', 'observedGeneration']), + ), + readyReplicas: numberOrNull( + getPath(deploymentJson, ['status', 'readyReplicas']), + ), + updatedReplicas: numberOrNull( + getPath(deploymentJson, ['status', 'updatedReplicas']), + ), + }; +} + +function scorePod( + pod: Record, + input: NormalizedInput, +): number { + const phase = stringOrNull(getPath(pod, ['status', 'phase'])); + const statuses = getArray(pod, ['status', 'containerStatuses']); + const status = findNamedObject(statuses, input.container); + const image = stringOrNull(status?.image); + return ( + (phase === 'Running' ? 4 : 0) + + (image && input.imageTag && image.includes(input.imageTag) ? 2 : 0) + + (status?.ready === true ? 1 : 0) + ); +} + +function normalizePodEvidence( + input: NormalizedInput, + podsJson: Record | null, +): DeployObservationPodEvidence { + const pods = getArray(podsJson, ['items']) + .map(objectAt) + .filter((item): item is Record => Boolean(item)) + .sort((left, right) => scorePod(right, input) - scorePod(left, input)); + const pod = pods[0] || null; + const status = findNamedObject( + getArray(pod, ['status', 'containerStatuses']), + input.container, + ); + const specContainer = findNamedObject( + getArray(pod, ['spec', 'containers']), + input.container, + ); + + return { + containerFound: Boolean(status || specContainer), + image: stringOrNull(status?.image) || stringOrNull(specContainer?.image), + name: stringOrNull(getPath(pod, ['metadata', 'name'])), + phase: stringOrNull(getPath(pod, ['status', 'phase'])), + ready: typeof status?.ready === 'boolean' ? status.ready : null, + restartCount: numberOrNull(status?.restartCount), + }; +} + +function normalizeRuntimeHealthEvidence( + healthJson: Record | null, +): DeployObservationRuntimeHealthEvidence { + const checks = getArray(healthJson, ['checks']); + + return { + checkCount: checks.length || null, + service: stringOrNull(healthJson?.service), + status: stringOrNull(healthJson?.status), + }; +} + +function parseSmokeExitCode(sectionMap: Map): number | null { + const status = readKeyValueSection(sectionMap.get('smoke_status') || ''); + if (!status.exitCode?.trim()) return null; + const exitCode = Number(status.exitCode); + return Number.isInteger(exitCode) && exitCode >= 0 ? exitCode : null; +} + +function buildAssertion( + name: string, + passed: boolean, + message: string, + critical = true, +): DeployObservationAssertion { + return { + critical, + message, + name, + passed, + }; +} + +function aggregateStatus( + assertions: DeployObservationAssertion[], +): DeployObservationStatus { + if (assertions.every((item) => item.passed)) return 'passed'; + if (assertions.some((item) => item.critical && !item.passed)) return 'failed'; + return 'blocked'; +} + +function toFilenameSlug(value: string): string { + const slug = value + .replace(/[^A-Za-z0-9_-]+/g, '-') + .replace(/^-+|-+$/g, '') + .slice(0, 80); + return slug || 'project'; +} + +export function normalizeDeployObservationEvidence({ + checkedAt = new Date(), + input, + sectionMap, +}: NormalizeEvidenceInput): DeployObservationEvidence { + const normalized = normalizeInput(input); + const jenkins = normalizeJenkinsEvidence(normalized, sectionMap); + const deployment = normalizeDeploymentEvidence( + normalized, + parseJsonObject(sectionMap.get('deployment_json') || ''), + ); + const pod = normalizePodEvidence( + normalized, + parseJsonObject(sectionMap.get('pods_json') || ''), + ); + const runtimeHealth = normalizeRuntimeHealthEvidence( + parseJsonObject(sectionMap.get('health_json') || ''), + ); + const smokeOutput = sectionMap.get('smoke_text') || ''; + const smokeExitCode = parseSmokeExitCode(sectionMap); + const expectedImageTag = normalized.imageTag || normalized.expectedCommit; + const deploymentObserved = + deployment.generation !== null && + deployment.observedGeneration !== null && + deployment.observedGeneration >= deployment.generation; + const deploymentReplicasReady = + deployment.updatedReplicas !== null && + deployment.readyReplicas !== null && + deployment.desiredReplicas !== null && + deployment.updatedReplicas >= deployment.desiredReplicas && + deployment.readyReplicas >= deployment.desiredReplicas; + const runtimeHealthStatusAllowed = + runtimeHealth.status === 'ready' || + runtimeHealth.status === 'live' || + runtimeHealth.status === 'degraded'; + const smokeSuccess = Boolean(normalized.smoke && smokeExitCode === 0); + const assertions = [ + buildAssertion( + 'jenkins-build-success', + jenkins.finishedStatus === 'SUCCESS', + `Jenkins ${jenkins.jobName} #${jenkins.buildNumber || 'unknown'} result=${jenkins.finishedStatus || 'missing'}`, + ), + buildAssertion( + 'jenkins-commit-match', + !jenkins.expectedCommit || jenkins.commitMatched === true, + jenkins.expectedCommit + ? `Expected commit ${jenkins.expectedCommit} matched=${jenkins.commitMatched}` + : 'No expected commit supplied', + Boolean(jenkins.expectedCommit), + ), + buildAssertion( + 'deployment-container-found', + deployment.containerFound, + `Deployment container=${deployment.container} found=${deployment.containerFound}`, + ), + buildAssertion( + 'deployment-image-tag', + Boolean( + deployment.image && + (!expectedImageTag || deployment.image.includes(expectedImageTag)), + ), + `Deployment image=${deployment.image || 'missing'}`, + ), + buildAssertion( + 'deployment-generation-observed', + deploymentObserved, + `Deployment generation=${deployment.generation} observedGeneration=${deployment.observedGeneration}`, + ), + buildAssertion( + 'deployment-replicas-ready', + deploymentReplicasReady, + `Deployment desired=${deployment.desiredReplicas} ready=${deployment.readyReplicas} updated=${deployment.updatedReplicas}`, + ), + buildAssertion( + 'pod-container-found', + pod.containerFound, + `Pod container=${normalized.container} found=${pod.containerFound}`, + ), + buildAssertion( + 'pod-running', + pod.phase === 'Running' && pod.ready === true, + `Pod ${pod.name || 'missing'} phase=${pod.phase || 'missing'} ready=${pod.ready}`, + ), + buildAssertion( + 'pod-image-tag', + Boolean(pod.image && (!expectedImageTag || pod.image.includes(expectedImageTag))), + `Pod image=${pod.image || 'missing'}`, + ), + buildAssertion( + 'pod-restart-count-zero', + pod.restartCount === 0, + `Pod restartCount=${pod.restartCount}`, + ), + buildAssertion( + 'runtime-health-available', + runtimeHealth.service === 'kt-template-online-api' && + runtimeHealthStatusAllowed, + `Runtime health status=${runtimeHealth.status || 'missing'}`, + ), + buildAssertion( + 'task-smoke-success', + smokeSuccess, + normalized.smoke + ? `Task smoke exitCode=${smokeExitCode ?? 'missing'}` + : 'No task smoke command supplied; rollout evidence is not functional evidence', + ), + ]; + + return { + assertions, + details: { + deployment, + eventsTail: sectionMap.get('events_tail') || '', + jenkins, + pod, + runtimeHealth, + smoke: { + command: normalized.smoke || null, + exitCode: smokeExitCode, + output: smokeOutput, + success: smokeSuccess, + }, + }, + endedAt: checkedAt.toISOString(), + environment: 'production', + operation: 'kt_deploy_observation', + project: normalized.project, + schemaVersion: 1, + startedAt: checkedAt.toISOString(), + status: aggregateStatus(assertions), + target: `${normalized.namespace}/${normalized.deployment}`, + taskType: 'deploy', + title: 'KT API deployment observation', + }; +} + +function writeEvidenceArtifact( + artifactRoot: string, + evidence: DeployObservationEvidence, +): string { + const root = resolveInsideRoot(artifactRoot); + const directory = path.join(root, formatDateInShanghai()); + const safeProject = toFilenameSlug(evidence.project); + const filePath = path.resolve( + directory, + `${safeProject}-${Date.now()}-deploy-observation.json`, + ); + const relative = path.relative(root, filePath); + if (relative.startsWith('..') || path.isAbsolute(relative)) { + throw new Error( + `Deploy observation artifact path escaped artifact root: ${filePath}`, + ); + } + + mkdirSync(directory, { recursive: true }); + writeFileSync(filePath, `${JSON.stringify(evidence, null, 2)}\n`, 'utf8'); + return filePath; +} + +export async function buildDeployObservation( + input: DeployObservationInput = {}, +): Promise { + const normalized = normalizeInput(input); + const commands = buildDeployObservationCommands(normalized); + const requiredEvidence = [ + 'Jenkins job/build number and final Finished status', + 'Expected commit hash match when a commit is supplied', + 'K8s Deployment target container, desiredReplicas, updatedReplicas, readyReplicas, generation, observedGeneration', + 'Running Pod selected by target container/current image tag with restartCount', + 'GET /health/runtime response with service/status/check count', + 'Task-specific smoke command output', + ]; + const notes = [ + 'Default execute=false only returns the bounded read-only command.', + 'execute=true runs through ssh nas with CRLF-stripped here-string and writes local JSON evidence.', + 'Jenkins/K8s rollout evidence is deployment evidence; task smoke is still required for functional completion.', + 'The command reads Jenkins build files, kubectl status, events, and HTTP health only; it does not read Secrets or mutate remote state.', + ]; + + if (!normalized.execute) { + return { + commands, + execute: false, + notes, + requiredEvidence, + }; + } + + const command = commands[0]?.command || ''; + const result = await tryPowerShell(command, workspaceRoot, 180_000); + const sectionMap = buildDeployObservationSectionMap(result.stdout); + const evidence = normalizeDeployObservationEvidence({ + input: normalized, + sectionMap, + }); + const artifactPath = writeEvidenceArtifact(normalized.artifactRoot, evidence); + + return { + artifactPath, + commands, + evidence, + execute: true, + notes, + requiredEvidence, + result, + }; +} diff --git a/src/tools/review.ts b/src/tools/review.ts index 70f42fb..66a3b7d 100644 --- a/src/tools/review.ts +++ b/src/tools/review.ts @@ -505,6 +505,26 @@ export function findQqbotNapcatCaptchaFlowFindings( }); } + if ( + content.includes("needNewDevice") && + (!content.includes("keepDeviceVerifyPending") || + !content.includes("deviceVerifyUrl")) + ) { + findings.push({ + category: "qqbot-napcat-new-device-drops-pending", + file: context.file, + line: + lines.findIndex((line) => line.includes("needNewDevice")) + 1 || + undefined, + level: "P2", + message: + "NapCat 验证码通过后仍可能要求新设备验证,直接提示或失败会丢掉可继续的 jumpUrl。", + project: context.project, + suggestion: + "CaptchaLogin 返回 needNewDevice/jumpUrl 时,应保持同一 scan session pending,返回 deviceVerifyUrl,清掉旧 captchaUrl,并继续轮询同一容器。", + }); + } + const logAnchorMatch = passwordReloginBody.match( /const\s+([A-Za-z0-9_]*LogSinceMs)\s*=\s*Date\.now\(\)[\s\S]*?ensureRuntimeLoginEnv/, ); diff --git a/src/types.ts b/src/types.ts index 31274a3..9b0711c 100644 --- a/src/types.ts +++ b/src/types.ts @@ -205,6 +205,116 @@ export interface RemoteHealthCheckInput { sshTarget?: string; } +export type DeployObservationStatus = + | 'passed' + | 'failed' + | 'blocked' + | 'skipped'; + +export interface DeployObservationInput { + artifactRoot?: string; + buildNumber?: string; + container?: string; + deployment?: string; + execute?: boolean; + expectedCommit?: string; + healthUrl?: string; + imageTag?: string; + jobName?: string; + jenkinsHome?: string; + kubeconfigPath?: string; + namespace?: string; + project?: string; + selector?: string; + smoke?: string; + sshPort?: number; + sshTarget?: string; +} + +export interface DeployObservationAssertion { + critical: boolean; + message: string; + name: string; + passed: boolean; +} + +export interface DeployObservationCommand { + command: string; + name: string; +} + +export interface DeployObservationJenkinsEvidence { + buildNumber: string | null; + commitMatched: boolean | null; + expectedCommit: string | null; + finishedStatus: string | null; + jobName: string; +} + +export interface DeployObservationDeploymentEvidence { + container: string; + containerFound: boolean; + desiredReplicas: number | null; + generation: number | null; + image: string | null; + namespace: string; + observedGeneration: number | null; + readyReplicas: number | null; + updatedReplicas: number | null; +} + +export interface DeployObservationPodEvidence { + containerFound: boolean; + image: string | null; + name: string | null; + phase: string | null; + ready: boolean | null; + restartCount: number | null; +} + +export interface DeployObservationRuntimeHealthEvidence { + checkCount: number | null; + service: string | null; + status: string | null; +} + +export interface DeployObservationEvidence { + assertions: DeployObservationAssertion[]; + details: { + deployment: DeployObservationDeploymentEvidence; + eventsTail: string; + jenkins: DeployObservationJenkinsEvidence; + pod: DeployObservationPodEvidence; + runtimeHealth: DeployObservationRuntimeHealthEvidence; + smoke: { + command: string | null; + exitCode: number | null; + output: string; + success: boolean; + }; + }; + endedAt: string; + environment: 'production'; + operation: 'kt_deploy_observation'; + project: string; + schemaVersion: 1; + startedAt: string; + status: DeployObservationStatus; + target: string; + taskType: 'deploy'; + title: string; +} + +export interface DeployObservationResult { + artifactPath?: string; + commands: DeployObservationCommand[]; + evidence?: DeployObservationEvidence; + execute: boolean; + notes: string[]; + requiredEvidence: string[]; + result?: ExecResult; +} + export interface DbSyncPlanInput { backupName?: string; source?: 'local' | 'remote';