docs: 补充环境总览MQTT事件层方案

2026-06-18 15:47:57 +08:00 · 2026-06-18 15:47:57 +08:00 · 6b8b76b6ff
commit 6b8b76b6ff
parent b9acee5af0
2 changed files with 130 additions and 3 deletions
--- a/docs/superpowers/plans/2026-06-18-admin-environment-dashboard-implementation-plan.md
+++ b/docs/superpowers/plans/2026-06-18-admin-environment-dashboard-implementation-plan.md
@ -4,9 +4,9 @@

 **Goal:** 将 Admin `/dashboard/analytics` 改造成多站点环境状态总览总控面板，覆盖本机开发、NAS 线上、腾讯云、r4se 远程环境，并明确展示未接入证据，第一版只做观测与只读自检，不提供高风险写操作。

-**Architecture:** 后端在 API `admin/platform-config` 下新增环境面板聚合模块，以 `Site -> Node -> Service -> Signal` 为统一模型；内部服务信号从现有 Nest 服务读取，远程信号通过只读适配器读取 Jenkins、Kubernetes、Tencent Cloud、Caddy、WireGuard、OpenClash/Mihomo；前端保留 Vben Dashboard 路由，替换为 A 方案布局：顶部状态条、左侧站点栏、中间拓扑、右侧证据/动作抽屉、底部事件流。
+**Architecture:** 后端在 API `admin/platform-config` 下新增环境面板聚合模块，以 `Site -> Node -> Service -> Signal` 为统一模型；HTTP dashboard/self-check 负责状态快照和只读兜底，平台级 EnvironmentEventBus 通过 local/mqtt 模式收口 topic 事件并驱动 recent events 与 cache invalidation；内部服务信号从现有 Nest 服务读取，远程信号通过只读适配器读取 Jenkins、Kubernetes、Tencent Cloud、Caddy、WireGuard、OpenClash/Mihomo；前端保留 Vben Dashboard 路由，替换为 A 方案布局：顶部状态条、左侧站点栏、中间拓扑、右侧证据/动作抽屉、底部事件流。

-**Tech Stack:** NestJS、TypeScript、TypeORM、axios、tencentcloud-sdk-nodejs、Vben Admin、Vue 3、antdv-next、Vitest/Jest、Playwright 轻量页面烟测。
+**Tech Stack:** NestJS、TypeScript、TypeORM、axios、mqtt、tencentcloud-sdk-nodejs、Vben Admin、Vue 3、antdv-next、Vitest/Jest、Playwright 轻量页面烟测。

 ---

@ -23,11 +23,15 @@
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\domain\environment-dashboard.types.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\application\environment-dashboard-status.mapper.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\application\environment-dashboard-action.catalog.ts`
+- `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\application\environment-event.materializer.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\application\environment-dashboard.service.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\application\environment-dashboard-self-check.service.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\environment-dashboard-config.service.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\environment-dashboard-cache.service.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\environment-dashboard-evidence.mapper.ts`
+- `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\event\environment-event-bus.service.ts`
+- `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\event\environment-mqtt-topic.catalog.ts`
+- `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\event\qqbot-environment-event.bridge.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\adapters\environment-readonly-http.client.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\adapters\jenkins-readonly.adapter.ts`
 - `D:\MyFiles\KT\Node\kt-template-online-api\src\modules\admin\platform-config\environment-dashboard\infrastructure\adapters\kubernetes-readonly.adapter.ts`
@ -95,6 +99,22 @@ export interface EnvironmentDashboardResponse {
  actions: EnvironmentAction[];
  events: EnvironmentEvent[];
 }
+
+export interface EnvironmentEventEnvelope {
+  eventId: string;
+  topic: string;
+  siteId: string;
+  nodeId?: string;
+  serviceId?: string;
+  signalId?: string;
+  severity: EnvironmentHealthStatus;
+  sourceKind: 'local' | 'mqtt' | EnvironmentSignalSourceKind;
+  observedAt: string;
+  expiresAt?: string;
+  retained?: boolean;
+  summary: string;
+  evidence?: EnvironmentEvidence[];
+}
 ```

 Severity aggregation order is fixed:
@ -118,6 +138,8 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
 - Every new or touched function/method/hook/event handler/exported arrow function must include JSDoc explaining purpose, parameter origin, return semantics, and side effects when present.
 - High-risk actions are represented by disabled action records with `enabled: false` and `disabledReason`; controller must reject execution paths because this plan does not add a generic write-action endpoint.
 - `unwired` and `unknown` are valid product states. A missing integration must be visible evidence, not a green health badge.
+- MQTT is an event layer, not the only source of truth. Retained messages require `observedAt` and `expiresAt`; expired retained messages must become `unknown` or `unwired`.
+- Admin must not connect to MQTT directly. Broker credentials and internal topics stay inside API runtime.
 - For interface changes, verify through a real local HTTP request against the Nest app or an already running local service.

 ---
@ -163,6 +185,13 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
  ```
  Expected output: dependency added to `package.json` and lockfile. No credential values are written.

+- [ ] Confirm the existing MQTT client dependency before adding any event-bus code:
+  ```powershell
+  cd D:\MyFiles\KT\Node\kt-template-online-api
+  rg -n '"mqtt"' package.json pnpm-lock.yaml
+  ```
+  Expected output: `mqtt` is already present in `package.json`; do not add a second broker/client package.
+
 ## Task 1: API Contract, Status Mapper, And RED Tests

 - [ ] Create `environment-dashboard.types.ts` with domain interfaces for site/node/service/signal/action/event/topology. Include status unions from the spec and source metadata:
@ -257,6 +286,43 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
  ```
  Expected output: tests pass; no real network access is required because axios is mocked.

+## Task 3A: API Platform Event Bus And MQTT Topic Contract
+
+- [ ] Create `infrastructure/event/environment-mqtt-topic.catalog.ts` with documented topic builders:
+  - `signal(siteId, nodeId, serviceId)`
+  - `event(siteId, nodeId, serviceId)`
+  - `selfCheckResult(siteId)`
+  - `qqbotRuntime(selfId)`
+  - `qqbotNapcatLogin(selfId)`
+  - `pluginTaskRun(pluginKey, taskKey)`
+
+- [ ] Create `infrastructure/event/environment-event-bus.service.ts`:
+  - supports `ENV_DASHBOARD_EVENT_BUS=local|mqtt`
+  - uses `ENV_DASHBOARD_MQTT_URL`, `ENV_DASHBOARD_MQTT_CLIENT_ID`, `ENV_DASHBOARD_MQTT_USERNAME`, `ENV_DASHBOARD_MQTT_PASSWORD`, and `ENV_DASHBOARD_MQTT_TOPIC_PREFIX`
+  - subscribes only to the environment dashboard topic prefix
+  - publishes local events through in-process subscribers for tests and dev
+  - marks broker disconnect as an environment event
+  - never logs broker credentials or full payloads
+
+- [ ] Create `application/environment-event.materializer.ts`:
+  - accepts `EnvironmentEventEnvelope`
+  - rejects stale retained messages when `expiresAt` is earlier than current time
+  - appends safe recent events for dashboard response
+  - invalidates `environment-dashboard-cache.service.ts` when a non-stale signal event arrives
+  - never turns MQTT-only data into green status without fresh `observedAt` evidence
+
+- [ ] Create `infrastructure/event/qqbot-environment-event.bridge.ts`:
+  - maps QQBot/NapCat runtime events into environment event envelopes
+  - depends on a narrow bus interface, not QQBot topic constants inside dashboard application code
+  - does not make environment dashboard depend on `QqbotBusService` implementation details
+
+- [ ] Add RED/GREEN tests:
+  ```powershell
+  cd D:\MyFiles\KT\Node\kt-template-online-api
+  pnpm exec jest --runTestsByPath test/modules/admin/environment-dashboard/environment-mqtt-topic.catalog.spec.ts test/modules/admin/environment-dashboard/environment-event-bus.service.spec.ts test/modules/admin/environment-dashboard/environment-event.materializer.spec.ts test/modules/admin/environment-dashboard/qqbot-environment-event.bridge.spec.ts --runInBand
+  ```
+  Expected output after implementation: topic mapping is deterministic, expired retained signal becomes non-green, broker disconnect creates an event without marking every service down, and QQBot bridge tests pass without importing dashboard code from QQBot-specific topic constants.
+
 ## Task 4: API Remote Adapters For Jenkins And Kubernetes

 - [ ] Create `environment-dashboard-config.service.ts` that reads optional dashboard integration env vars. It returns explicit missing-key lists for each integration instead of throwing.
@ -365,6 +431,7 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
 - [ ] Create `environment-dashboard.service.ts`:
  - assembles four sites: `local-dev`, `nas-prod`, `tencent-cloud`, `r4se`
  - aggregates summary counters
+  - merges safe recent events from `environment-event.materializer.ts`
  - builds topology edges:
    - local-dev -> API/Admin local services
    - nas-prod -> Jenkins/K8s/API/Admin/MySQL/Redis/Loki/MinIO/WordPress/QQBot/NapCat/Plugin Platform/Plugin Tasks
@ -376,7 +443,7 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
 - [ ] Create `environment-dashboard-self-check.service.ts`:
  - runs the same collectors with `forceRefresh: true`
  - returns `EnvironmentDashboardResponse`
-  - records event entries for failed adapters
+  - records event entries for failed adapters through `EnvironmentEventBus`
  - performs no write operation

 - [ ] Add service tests:
@ -403,6 +470,12 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
  ```env
  ENV_DASHBOARD_CACHE_TTL_MS=15000
  ENV_DASHBOARD_SIGNAL_TIMEOUT_MS=5000
+  ENV_DASHBOARD_EVENT_BUS=local
+  ENV_DASHBOARD_MQTT_URL=
+  ENV_DASHBOARD_MQTT_CLIENT_ID=kt-template-online-api-environment
+  ENV_DASHBOARD_MQTT_USERNAME=
+  ENV_DASHBOARD_MQTT_PASSWORD=
+  ENV_DASHBOARD_MQTT_TOPIC_PREFIX=kt/env
  ENV_DASHBOARD_JENKINS_URL=
  ENV_DASHBOARD_JENKINS_JOB=KT-Template/KT-Template-API/main
  ENV_DASHBOARD_JENKINS_USERNAME=
@ -543,6 +616,7 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
  - mocked API returns all four sites
  - disabled write actions render disabled
  - unwired Jenkins/K8s evidence renders as evidence, not healthy state
+  - MQTT-origin recent event renders only after API returns it; page does not instantiate an MQTT client
  - self-check button calls the POST wrapper

 - [ ] Run:
@ -569,6 +643,7 @@ const severityWeight: Record<EnvironmentHealthStatus, number> = {
  - topology contains Jenkins, K8s, QQBot/NapCat, Plugin Tasks, Tencent Caddy/WireGuard, r4se OpenClash/WireGuard
  - right panel shows evidence for selected item
  - bottom event stream appears
+  - event stream can display an API-provided MQTT-origin event without exposing broker configuration in frontend state
  - Jenkins/K8s missing config appears as `unwired` or `unknown`, never green
  - disabled write actions cannot be clicked

@ -657,6 +732,8 @@ Online completion evidence must include:

 - `GET /system/environment/dashboard` returns four sites.
 - `POST /system/environment/self-check` returns fresh evidence.
+- MQTT event bus is either `local` with explicit configured evidence, or `mqtt` with broker connection status shown as evidence.
+- A synthetic non-secret MQTT signal event invalidates dashboard cache and appears in recent events; an expired retained signal does not create a green status.
 - Jenkins/K8s nodes show live evidence when configured, or explicit missing-key evidence when not configured.
 - Tencent Cloud and r4se nodes do not show green health without a live/configured source.
 - Admin page displays the same state without console errors.
--- a/docs/superpowers/specs/2026-06-18-admin-environment-dashboard-design.md
+++ b/docs/superpowers/specs/2026-06-18-admin-environment-dashboard-design.md
@ -63,6 +63,7 @@ Admin 当前 `/dashboard/analytics` 仍是 Vben 示例分析页，展示用户
 9. Jenkins/K8s 必须接入只读观测；缺配置、403、超时或网络失败时显示失败证据，不能伪造成健康。
 10. 腾讯云/r4se 第一版以只读观测为目标；无法直接观测的 WireGuard 全局 peer 状态必须明确显示证据缺口，不抓取私有 WebUI。
 11. Figma 官方写入不作为本轮主链路；页面布局通过本地 visual companion 和中文 spec 收敛，避免 Figma 官方限流。
+12. 事件层采用 HTTP 快照 + MQTT 收口的混合方案：`GET /system/environment/dashboard` 和 `POST /system/environment/self-check` 继续作为状态真相源与兜底；MQTT 只负责 topic 事件订阅、recent event materialize 和 cache invalidation。Admin 前端不直连 MQTT。

 ## 目标

@ -70,6 +71,7 @@ Admin 当前 `/dashboard/analytics` 仍是 Vben 示例分析页，展示用户
 - 提供统一 API 合同，输出站点、摘要卡、拓扑节点、拓扑边、动作目录和事件流。
 - 每个节点必须带证据：来源、检查时间、实时/缓存/推导状态、失败原因、是否脱敏。
 - 让 Jenkins/K8s、腾讯云、r4se 不再是“未接入健康假象”，而是明确展示只读证据或缺口。
+- 引入平台级 MQTT 事件入口，统一收口 QQBot/NapCat、Plugin Task、Jenkins/K8s、腾讯云、Caddy、WireGuard、Mihomo/OpenClash 等运行态事件。
 - 后续可扩展远程站点探针，但第一版不引入写权限和高风险远程控制。

 ## 非目标
@ -79,6 +81,8 @@ Admin 当前 `/dashboard/analytics` 仍是 Vben 示例分析页，展示用户
 - 不读取 K8s Secret、Jenkins console 全量日志、QQBot 大字段、原始 env、token、kubeconfig、SSH key。
 - 不做营销式 hero、装饰图表、大面积单色主题或静态展示页。
 - 不新增长期后台巡检表或历史趋势表，除非实施计划后续单独确认。
+- 不让 MQTT 成为唯一状态真相源；broker 断连、retained message 过期或远程 agent 未接入时，dashboard 必须回退到 HTTP 快照/只读自检和明确缺口证据。
+- 不让 Admin 浏览器直接连接 MQTT broker，不向前端暴露 broker 用户名、密码、token 或内部 topic。

 ## 页面布局

@ -227,6 +231,30 @@ type EnvironmentEdgeRelation =
 - 跳转相关页面。
 - 脱敏摘要展示。

+### MQTT 事件收口
+
+环境面板使用平台级 `EnvironmentEventBus` 收口事件。它可以运行在 `local` 或 `mqtt` 模式：`local` 用于本地测试和无 broker 环境，`mqtt` 用于线上和远程探针接入。现有 QQBot MQTT 能力只能作为事件来源之一，通过桥接映射为环境事件；环境面板不得直接依赖 QQBot 专用 topic 常量或 `QqbotBusService`。
+
+建议 topic 前缀：
+
+```text
+kt/env/{siteKey}/{nodeKey}/{serviceKey}/signal
+kt/env/{siteKey}/{nodeKey}/{serviceKey}/event
+kt/env/{siteKey}/self-check/result
+kt/qqbot/{selfId}/runtime/event
+kt/qqbot/{selfId}/napcat/login
+kt/plugin/{pluginKey}/task/{taskKey}/run
+```
+
+MQTT 语义：
+
+- `signal` 可以使用 retained message，但 payload 必须包含 `observedAt` 和 `expiresAt`。过期 retained message 只能显示为 `unknown` 或 `unwired`，不能显示绿色健康。
+- 普通运行事件不使用 retained message，只进入 recent event materializer。
+- 状态信号和关键任务结果使用 QoS 1；普通日志摘要事件使用 QoS 0。
+- payload 入库、日志、API 返回前三层都要脱敏。
+- broker 断连是一个 `EnvironmentEvent`，同时使 dashboard cache 失效；不能因为 MQTT 断连把所有服务判定为 down。
+- Admin 不直连 MQTT；前端通过 dashboard HTTP 接口、self-check 或后续 API SSE/WebSocket 读取已经脱敏和聚合后的状态。
+
 ## API 合同

 新增受保护接口：
@ -251,10 +279,15 @@ src/modules/admin/platform-config/environment-dashboard/
    environment-dashboard-site.mapper.ts
    environment-dashboard-status.mapper.ts
    environment-dashboard-action.catalog.ts
+    environment-event.materializer.ts
  domain/
    environment-dashboard.types.ts
  infrastructure/
    environment-dashboard-signal.collector.ts
+    event/
+      environment-event-bus.service.ts
+      environment-mqtt-topic.catalog.ts
+      qqbot-environment-event.bridge.ts
    adapters/
      jenkins-readonly.adapter.ts
      kubernetes-readonly.adapter.ts
@ -292,6 +325,22 @@ interface EnvironmentDashboardDto {
  recentEvents: EnvironmentEventDto[];
 }

+interface EnvironmentEventEnvelopeDto {
+  eventId: string;
+  topic: string;
+  siteKey: string;
+  nodeKey?: string;
+  serviceKey?: string;
+  signalKey?: string;
+  severity: EnvironmentHealthStatus;
+  sourceKind: 'mqtt' | 'local' | EnvironmentSignalSourceKind;
+  observedAt: string;
+  expiresAt?: string;
+  retained?: boolean;
+  summary: string;
+  evidence?: EnvironmentEvidenceDto[];
+}
+
 interface EnvironmentSiteDto {
  key: string;
  title: string;
@ -897,6 +946,7 @@ EnvironmentDashboardPage

 - API service 单测：状态严重度聚合、site 聚合、partial failure、isolated/unknown/unwired、action catalog。
 - API adapter 单测：Jenkins/K8s/Tencent/Caddy/Mihomo/WireGuard 的 200、403、timeout、unreachable、failed/degraded mapping。
+- API event bus 单测：MQTT topic mapping、retained message expiry、broker disconnect event、QQBot bridge 脱耦。
 - API controller contract spec：鉴权路由、Vben response shape、self-check 只读行为。
 - Admin API wrapper Vitest：请求路径和响应类型。
 - Admin 页面组件测试：站点导航、拓扑节点、证据抽屉、disabled actions、事件流过滤。