feat: 建立 SAM2 标注闭环基线

- 打通工作区真实标注闭环：支持手工多边形、矩形、圆形、点区域和线段生成 mask，并可保存、回显、更新和删除后端 annotation。 - 增强 polygon 编辑器：支持顶点拖动、顶点删除、边中点插入、多 polygon 子区域选择编辑，以及区域合并和区域去除。 - 接入 GT mask 导入：后端支持二值/多类别 mask 拆分、contour 转 polygon、distance transform seed point，前端支持导入、回显和 seed point 拖动编辑。 - 完善导出能力：COCO JSON 导出对齐前端，PNG mask ZIP 同时包含单标注 mask、按 zIndex 融合的 semantic_frame 和 semantic_classes.json。 - 打通异步任务管理：新增任务取消、重试、失败详情接口与 Dashboard 控件，worker 支持取消状态检查并通过 Redis/WebSocket 推送 cancelled 事件。 - 对接 Dashboard 后端数据：概览统计、解析队列和实时流转记录从 FastAPI 聚合接口与 WebSocket 更新。 - 增强 AI 推理参数：前端发送 crop_to_prompt、auto_filter_background 和 min_score，后端支持点/框 prompt 局部裁剪推理、结果回映射和负向点/低分过滤。 - 接入 SAM3 基础设施：新增独立 Python 3.12 sam3 环境安装脚本、外部 worker helper、后端桥接和真实 Python/CUDA/包/HF checkpoint access 状态检测。 - 保留 SAM3 授权边界：当前官方 facebook/sam3 gated 权重未授权时状态接口会返回不可用，不伪装成可推理。 - 增强前端状态管理：新增 mask undo/redo 历史栈、AI 模型选择状态、保存状态 dirty/draft/saved 流转和项目状态归一化。 - 更新前端 API 封装：补充 annotation CRUD、GT mask import、mask ZIP export、task cancel/retry/detail、AI runtime status 和 prediction options。 - 更新 UI 控件：ToolsPalette、AISegmentation、VideoWorkspace 和 CanvasArea 接入真实操作、导入导出、撤销重做、任务控制和模型状态。 - 新增 polygon-clipping 依赖，用于前端区域 union/difference 几何运算。 - 完善后端 schemas/status/progress：补充 AI 模型外部状态字段、任务 cancelled 状态和进度事件 payload。 - 补充测试覆盖：新增后端任务控制、SAM3 桥接、GT mask、导出融合、AI options 测试；补充前端 Canvas、Dashboard、VideoWorkspace、ToolsPalette、API 和 store 测试。 - 更新 README、AGENTS 和 doc 文档：冻结当前需求/设计/测试计划，标注真实功能、剩余 Mock、SAM3 授权边界和后续实施顺序。
2026-05-01 15:26:25 +08:00
parent f020ff3b4f
commit 689a9ba283
48 changed files with 3280 additions and 176 deletions
--- a/src/components/CanvasArea.test.tsx
+++ b/src/components/CanvasArea.test.tsx
@@ -79,6 +79,271 @@ describe('CanvasArea', () => {
    expect(screen.getByText('遮罩数: 1')).toBeInTheDocument();
  });

+  it('renders imported GT seed points for editable point regions', () => {
+    useStore.setState({
+      masks: [
+        {
+          id: 'gt-1',
+          frameId: 'frame-1',
+          pathData: 'M 0 0 L 10 0 L 10 10 Z',
+          label: 'GT',
+          color: '#22c55e',
+          points: [[120, 80]],
+        },
+      ],
+    });
+
+    render(<CanvasArea activeTool="move" frame={frame} />);
+
+    expect(screen.getAllByTestId('konva-circle')).toHaveLength(2);
+  });
+
+  it('selects a polygon mask and drags a vertex into dirty saved state', () => {
+    useStore.setState({
+      masks: [
+        {
+          id: 'annotation-99',
+          annotationId: '99',
+          frameId: 'frame-1',
+          pathData: 'M 10 10 L 90 10 L 90 40 Z',
+          label: 'Saved',
+          color: '#06b6d4',
+          saved: true,
+          saveStatus: 'saved',
+          segmentation: [[10, 10, 90, 10, 90, 40]],
+          bbox: [10, 10, 80, 30],
+        },
+      ],
+    });
+
+    render(<CanvasArea activeTool="move" frame={frame} />);
+    fireEvent.click(screen.getByTestId('konva-path'));
+    const handles = screen.getAllByTestId('konva-circle')
+      .filter((element) => element.getAttribute('data-fill') === '#ffffff');
+    expect(handles).toHaveLength(3);
+
+    fireEvent.mouseUp(handles[0], { clientX: 20, clientY: 30 });
+
+    expect(useStore.getState().masks[0]).toEqual(expect.objectContaining({
+      pathData: 'M 20 30 L 90 10 L 90 40 Z',
+      segmentation: [[20, 30, 90, 10, 90, 40]],
+      bbox: [20, 10, 70, 30],
+      area: 1050,
+      saveStatus: 'dirty',
+      saved: false,
+    }));
+  });
+
+  it('deletes a selected polygon vertex without dropping below three points', () => {
+    useStore.setState({
+      masks: [
+        {
+          id: 'draft-1',
+          frameId: 'frame-1',
+          pathData: 'M 10 10 L 90 10 L 90 40 L 10 40 Z',
+          label: 'Draft',
+          color: '#06b6d4',
+          saveStatus: 'draft',
+          segmentation: [[10, 10, 90, 10, 90, 40, 10, 40]],
+          bbox: [10, 10, 80, 30],
+        },
+      ],
+    });
+
+    render(<CanvasArea activeTool="move" frame={frame} />);
+    fireEvent.click(screen.getByTestId('konva-path'));
+    const handles = screen.getAllByTestId('konva-circle')
+      .filter((element) => element.getAttribute('data-fill') === '#ffffff');
+    fireEvent.click(handles[0]);
+    fireEvent.keyDown(window, { key: 'Delete' });
+
+    expect(useStore.getState().masks[0]).toEqual(expect.objectContaining({
+      pathData: 'M 90 10 L 90 40 L 10 40 Z',
+      segmentation: [[90, 10, 90, 40, 10, 40]],
+      saveStatus: 'draft',
+    }));
+  });
+
+  it('inserts a polygon vertex from an edge midpoint handle', () => {
+    useStore.setState({
+      masks: [
+        {
+          id: 'draft-1',
+          frameId: 'frame-1',
+          pathData: 'M 10 10 L 90 10 L 90 40 Z',
+          label: 'Draft',
+          color: '#06b6d4',
+          saveStatus: 'draft',
+          segmentation: [[10, 10, 90, 10, 90, 40]],
+          bbox: [10, 10, 80, 30],
+        },
+      ],
+    });
+
+    render(<CanvasArea activeTool="move" frame={frame} />);
+    fireEvent.click(screen.getByTestId('konva-path'));
+    const edgeHandles = screen.getAllByTestId('konva-circle')
+      .filter((element) => element.getAttribute('data-fill') === '#22d3ee');
+    fireEvent.click(edgeHandles[0]);
+
+    expect(useStore.getState().masks[0]).toEqual(expect.objectContaining({
+      segmentation: [[10, 10, 50, 10, 90, 10, 90, 40]],
+      pathData: 'M 10 10 L 50 10 L 90 10 L 90 40 Z',
+      saveStatus: 'draft',
+    }));
+  });
+
+  it('edits the selected polygon in a multi-polygon mask', () => {
+    useStore.setState({
+      masks: [
+        {
+          id: 'multi-1',
+          frameId: 'frame-1',
+          pathData: 'M 10 10 L 50 10 L 50 40 Z M 100 100 L 150 100 L 150 140 Z',
+          label: 'Multi',
+          color: '#06b6d4',
+          saveStatus: 'draft',
+          segmentation: [
+            [10, 10, 50, 10, 50, 40],
+            [100, 100, 150, 100, 150, 140],
+          ],
+          bbox: [10, 10, 140, 130],
+        },
+      ],
+    });
+
+    render(<CanvasArea activeTool="move" frame={frame} />);
+    const paths = screen.getAllByTestId('konva-path');
+    fireEvent.click(paths[1]);
+    const vertexHandles = screen.getAllByTestId('konva-circle')
+      .filter((element) => element.getAttribute('data-fill') === '#ffffff');
+    fireEvent.mouseUp(vertexHandles[0], { clientX: 120, clientY: 120 });
+
+    expect(useStore.getState().masks[0].segmentation).toEqual([
+      [10, 10, 50, 10, 50, 40],
+      [120, 120, 150, 100, 150, 140],
+    ]);
+  });
+
+  it('merges selected draft masks with polygon union', () => {
+    useStore.setState({
+      masks: [
+        {
+          id: 'm1',
+          frameId: 'frame-1',
+          pathData: 'M 10 10 L 90 10 L 90 50 L 10 50 Z',
+          label: 'A',
+          color: '#06b6d4',
+          segmentation: [[10, 10, 90, 10, 90, 50, 10, 50]],
+        },
+        {
+          id: 'm2',
+          frameId: 'frame-1',
+          pathData: 'M 50 30 L 120 30 L 120 80 L 50 80 Z',
+          label: 'B',
+          color: '#ff0000',
+          segmentation: [[50, 30, 120, 30, 120, 80, 50, 80]],
+        },
+      ],
+    });
+
+    render(<CanvasArea activeTool="area_merge" frame={frame} />);
+    const paths = screen.getAllByTestId('konva-path');
+    fireEvent.click(paths[0]);
+    fireEvent.click(paths[1]);
+    fireEvent.click(screen.getByRole('button', { name: '合并选中' }));
+
+    expect(useStore.getState().masks).toHaveLength(1);
+    expect(useStore.getState().masks[0]).toEqual(expect.objectContaining({
+      id: 'm1',
+      segmentation: [[10, 10, 90, 10, 90, 30, 120, 30, 120, 80, 50, 80, 50, 50, 10, 50]],
+      bbox: [10, 10, 110, 70],
+      saveStatus: 'draft',
+    }));
+  });
+
+  it('removes overlap from the primary selected mask with polygon difference', () => {
+    useStore.setState({
+      masks: [
+        {
+          id: 'm1',
+          frameId: 'frame-1',
+          pathData: 'M 10 10 L 90 10 L 90 50 L 10 50 Z',
+          label: 'A',
+          color: '#06b6d4',
+          segmentation: [[10, 10, 90, 10, 90, 50, 10, 50]],
+        },
+        {
+          id: 'm2',
+          frameId: 'frame-1',
+          pathData: 'M 50 30 L 120 30 L 120 80 L 50 80 Z',
+          label: 'B',
+          color: '#ff0000',
+          segmentation: [[50, 30, 120, 30, 120, 80, 50, 80]],
+        },
+      ],
+    });
+
+    render(<CanvasArea activeTool="area_remove" frame={frame} />);
+    const paths = screen.getAllByTestId('konva-path');
+    fireEvent.click(paths[0]);
+    fireEvent.click(paths[1]);
+    fireEvent.click(screen.getByRole('button', { name: '从主区域去除' }));
+
+    expect(useStore.getState().masks).toHaveLength(2);
+    expect(useStore.getState().masks[0]).toEqual(expect.objectContaining({
+      id: 'm1',
+      segmentation: [[10, 10, 90, 10, 90, 30, 50, 30, 50, 50, 10, 50]],
+      bbox: [10, 10, 80, 40],
+      saveStatus: 'draft',
+    }));
+    expect(useStore.getState().masks[1].id).toBe('m2');
+  });
+
+  it('creates a manual rectangle mask that can be undone and redone', () => {
+    useStore.setState({
+      activeTemplateId: '2',
+      activeClass: { id: 'c1', name: '胆囊', color: '#ff0000', zIndex: 20 },
+      activeClassId: 'c1',
+    });
+
+    render(<CanvasArea activeTool="create_rectangle" frame={frame} />);
+    const stage = screen.getByTestId('konva-stage');
+    fireEvent.mouseDown(stage);
+    fireEvent.mouseMove(stage);
+    fireEvent.mouseUp(stage);
+
+    expect(useStore.getState().masks).toHaveLength(1);
+    expect(useStore.getState().masks[0]).toEqual(expect.objectContaining({
+      frameId: 'frame-1',
+      label: '胆囊',
+      color: '#ff0000',
+      saveStatus: 'draft',
+      segmentation: [[120, 80, 260, 80, 260, 200, 120, 200]],
+      bbox: [120, 80, 140, 120],
+    }));
+
+    useStore.getState().undoMasks();
+    expect(useStore.getState().masks).toEqual([]);
+    useStore.getState().redoMasks();
+    expect(useStore.getState().masks).toHaveLength(1);
+  });
+
+  it('finalizes a clicked polygon with Enter', () => {
+    render(<CanvasArea activeTool="create_polygon" frame={frame} />);
+    const stage = screen.getByTestId('konva-stage');
+    fireEvent.click(stage, { clientX: 120, clientY: 80 });
+    fireEvent.click(stage, { clientX: 220, clientY: 80 });
+    fireEvent.click(stage, { clientX: 180, clientY: 160 });
+    fireEvent.keyDown(window, { key: 'Enter' });
+
+    expect(useStore.getState().masks).toHaveLength(1);
+    expect(useStore.getState().masks[0].metadata).toEqual(expect.objectContaining({
+      source: 'manual',
+      shape: '多边形',
+    }));
+  });
+
  it('applies the selected class to current-frame masks and marks saved masks dirty', () => {
    useStore.setState({
      activeTemplateId: '2',