feat: 打通全栈标注闭环、异步拆帧与模型状态
后端能力： - 新增 Celery app、worker task、ProcessingTask 模型、/api/tasks 查询接口和 media_task_runner，将 /api/media/parse 改为创建后台任务并由 worker 执行 FFmpeg/OpenCV/pydicom 拆帧。 - 新增 Redis 进度事件模块和 FastAPI Redis pub/sub 订阅，将 worker 任务进度广播到 /ws/progress；Dashboard 后端概览接口改为聚合 projects/frames/annotations/templates/processing_tasks。 - 统一项目状态为 pending/parsing/ready/error，新增共享 status 常量，并让前端兼容归一化旧状态值。 - 扩展 AI 后端：新增 SAM registry、SAM2 真实运行状态、SAM3 状态检测与文本语义推理适配入口，以及 /api/ai/models/status GPU/模型状态接口。 - 补齐标注保存/更新/删除、COCO/PNG mask 导出相关后端契约和模板 mapping_rules 打包/解包行为。 前端能力： - 新增运行时 API/WS 地址推导配置，前端 API 封装对齐 FastAPI 路由、字段映射、任务轮询、标注归档、导出下载和 AI 预测响应转换。 - Dashboard 改为读取 /api/dashboard/overview，并订阅 WebSocket progress/complete/error/status 更新解析队列和实时流转记录。 - 项目库导入视频/DICOM 后创建项目、上传媒体、触发异步解析并刷新真实项目列表。 - 工作区加载真实帧、无帧时触发解析任务、回显已保存标注、保存未归档 mask、更新 dirty mask、清空当前帧后端标注、导出 COCO JSON。 - Canvas 支持当前帧点/框提示调用后端 AI、渲染推理/已保存 mask、应用模板分类并维护保存状态计数；时间轴按项目 fps 播放。 - AI 页面新增 SAM2/SAM3 模型选择，预测请求携带 model；侧边栏和工作区新增真实 GPU/SAM 状态徽标。 - 模板库和本体面板接入真实模板 CRUD、分类编辑、拖拽排序、JSON 导入、默认腹腔镜分类和本地自定义分类选择。 测试与文档： - 新增 Vitest 配置、前端测试 setup、API/config/websocket/store/组件测试，覆盖登录、项目库、Dashboard、Canvas、工作区、模型状态、时间轴、本体和模板库。 - 新增 pytest 后端测试夹具和 auth/projects/templates/media/AI/export/dashboard/tasks/progress 测试，使用 SQLite、fake MinIO、fake SAM registry 和 Redis monkeypatch 隔离外部服务。 - 新增 doc/ 文档结构，冻结当前需求、设计、接口契约、测试计划、前端逐元素审计、实现地图和后续实施计划，并同步更新 README 与 AGENTS。 验证： - conda run -n seg_server pytest backend/tests：27 passed。 - npm run test:run：54 passed。 - npm run lint、npm run build、compileall、git diff --check 均通过；Vite 仅提示大 chunk 警告。
This commit is contained in:
259
src/components/VideoWorkspace.test.tsx
Normal file
259
src/components/VideoWorkspace.test.tsx
Normal file
@@ -0,0 +1,259 @@
|
||||
import { act, fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { resetStore } from '../test/storeTestUtils';
|
||||
import { useStore } from '../store/useStore';
|
||||
import { VideoWorkspace } from './VideoWorkspace';
|
||||
|
||||
const apiMock = vi.hoisted(() => ({
|
||||
getProjectFrames: vi.fn(),
|
||||
parseMedia: vi.fn(),
|
||||
getTask: vi.fn(),
|
||||
getTemplates: vi.fn(),
|
||||
getProjectAnnotations: vi.fn(),
|
||||
saveAnnotation: vi.fn(),
|
||||
updateAnnotation: vi.fn(),
|
||||
deleteAnnotation: vi.fn(),
|
||||
exportCoco: vi.fn(),
|
||||
annotationToMask: vi.fn(),
|
||||
buildAnnotationPayload: vi.fn(),
|
||||
getAiModelStatus: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('../lib/api', () => ({
|
||||
getProjectFrames: apiMock.getProjectFrames,
|
||||
parseMedia: apiMock.parseMedia,
|
||||
getTask: apiMock.getTask,
|
||||
getTemplates: apiMock.getTemplates,
|
||||
getProjectAnnotations: apiMock.getProjectAnnotations,
|
||||
saveAnnotation: apiMock.saveAnnotation,
|
||||
updateAnnotation: apiMock.updateAnnotation,
|
||||
deleteAnnotation: apiMock.deleteAnnotation,
|
||||
exportCoco: apiMock.exportCoco,
|
||||
annotationToMask: apiMock.annotationToMask,
|
||||
buildAnnotationPayload: apiMock.buildAnnotationPayload,
|
||||
getAiModelStatus: apiMock.getAiModelStatus,
|
||||
}));
|
||||
|
||||
describe('VideoWorkspace', () => {
|
||||
beforeEach(() => {
|
||||
resetStore();
|
||||
vi.clearAllMocks();
|
||||
useStore.setState({ currentProject: { id: '1', name: 'Demo', status: 'ready', video_path: 'uploads/demo.mp4' } });
|
||||
apiMock.getTemplates.mockResolvedValue([]);
|
||||
apiMock.getProjectAnnotations.mockResolvedValue([]);
|
||||
apiMock.annotationToMask.mockReturnValue(null);
|
||||
apiMock.getTask.mockResolvedValue({ id: 1, status: 'success', progress: 100, message: '解析完成' });
|
||||
apiMock.getAiModelStatus.mockResolvedValue({
|
||||
selected_model: 'sam2',
|
||||
gpu: { available: false, device: 'cpu', name: null, torch_available: true },
|
||||
models: [
|
||||
{ id: 'sam2', label: 'SAM 2', available: true, loaded: false, device: 'cpu', supports: [], message: 'ready', package_available: true, checkpoint_exists: true, python_ok: true, torch_ok: true, cuda_required: false },
|
||||
{ id: 'sam3', label: 'SAM 3', available: false, loaded: false, device: 'unavailable', supports: [], message: 'missing', package_available: false, checkpoint_exists: false, python_ok: false, torch_ok: true, cuda_required: true },
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it('loads project frames into the workspace store', async () => {
|
||||
apiMock.getProjectFrames.mockResolvedValueOnce([
|
||||
{ id: 10, project_id: 1, frame_index: 0, image_url: '/frame.jpg', width: 640, height: 360 },
|
||||
]);
|
||||
|
||||
render(<VideoWorkspace />);
|
||||
|
||||
await waitFor(() => expect(useStore.getState().frames).toEqual([
|
||||
{ id: '10', projectId: '1', index: 0, url: '/frame.jpg', width: 640, height: 360 },
|
||||
]));
|
||||
expect(screen.getByText('Demo')).toBeInTheDocument();
|
||||
expect(apiMock.parseMedia).not.toHaveBeenCalled();
|
||||
expect(apiMock.getProjectAnnotations).toHaveBeenCalledWith('1');
|
||||
});
|
||||
|
||||
it('triggers parsing when a media project has no frames yet', async () => {
|
||||
apiMock.getProjectFrames
|
||||
.mockResolvedValueOnce([])
|
||||
.mockResolvedValueOnce([
|
||||
{ id: 11, project_id: 1, frame_index: 0, image_url: '/parsed.jpg', width: 320, height: 240 },
|
||||
]);
|
||||
apiMock.parseMedia.mockResolvedValueOnce({ id: 7, status: 'queued', progress: 0 });
|
||||
apiMock.getTask.mockResolvedValueOnce({ id: 7, status: 'success', progress: 100, message: '解析完成' });
|
||||
|
||||
render(<VideoWorkspace />);
|
||||
|
||||
await waitFor(() => expect(apiMock.parseMedia).toHaveBeenCalledWith('1'));
|
||||
expect(apiMock.getTask).toHaveBeenCalledWith(7);
|
||||
await waitFor(() => expect(useStore.getState().frames[0]).toEqual(expect.objectContaining({
|
||||
id: '11',
|
||||
url: '/parsed.jpg',
|
||||
})));
|
||||
});
|
||||
|
||||
it('hydrates saved annotations after loading frames', async () => {
|
||||
apiMock.getProjectFrames.mockResolvedValueOnce([
|
||||
{ id: 10, project_id: 1, frame_index: 0, image_url: '/frame.jpg', width: 640, height: 360 },
|
||||
]);
|
||||
apiMock.getProjectAnnotations.mockResolvedValueOnce([{ id: 99, frame_id: 10 }]);
|
||||
apiMock.annotationToMask.mockReturnValueOnce({
|
||||
id: 'annotation-99',
|
||||
annotationId: '99',
|
||||
frameId: '10',
|
||||
saved: true,
|
||||
pathData: 'M 0 0 Z',
|
||||
label: 'Saved',
|
||||
color: '#06b6d4',
|
||||
});
|
||||
|
||||
render(<VideoWorkspace />);
|
||||
|
||||
await waitFor(() => expect(useStore.getState().masks).toEqual([
|
||||
expect.objectContaining({ id: 'annotation-99', saved: true }),
|
||||
]));
|
||||
});
|
||||
|
||||
it('saves pending masks through the archive button', async () => {
|
||||
apiMock.getProjectFrames.mockResolvedValueOnce([
|
||||
{ id: 10, project_id: 1, frame_index: 0, image_url: '/frame.jpg', width: 640, height: 360 },
|
||||
]);
|
||||
apiMock.buildAnnotationPayload.mockReturnValueOnce({ project_id: 1, frame_id: 10, mask_data: { polygons: [] } });
|
||||
apiMock.saveAnnotation.mockResolvedValueOnce({ id: 5 });
|
||||
|
||||
render(<VideoWorkspace />);
|
||||
await waitFor(() => expect(useStore.getState().frames).toHaveLength(1));
|
||||
act(() => {
|
||||
useStore.setState({
|
||||
activeTemplateId: '2',
|
||||
masks: [{
|
||||
id: 'mask-1',
|
||||
frameId: '10',
|
||||
pathData: 'M 0 0 Z',
|
||||
label: 'AI Mask',
|
||||
color: '#06b6d4',
|
||||
segmentation: [[0, 0, 10, 0, 10, 10]],
|
||||
bbox: [0, 0, 10, 10],
|
||||
}],
|
||||
});
|
||||
});
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '结构化归档保存' }));
|
||||
|
||||
await waitFor(() => expect(apiMock.saveAnnotation).toHaveBeenCalledWith({
|
||||
project_id: 1,
|
||||
frame_id: 10,
|
||||
mask_data: { polygons: [] },
|
||||
}));
|
||||
expect(apiMock.buildAnnotationPayload).toHaveBeenCalledWith(
|
||||
'1',
|
||||
expect.objectContaining({ id: 'mask-1' }),
|
||||
expect.objectContaining({ id: '10' }),
|
||||
'2',
|
||||
);
|
||||
});
|
||||
|
||||
it('updates dirty saved masks through the archive button', async () => {
|
||||
apiMock.getProjectFrames.mockResolvedValueOnce([
|
||||
{ id: 10, project_id: 1, frame_index: 0, image_url: '/frame.jpg', width: 640, height: 360 },
|
||||
]);
|
||||
apiMock.buildAnnotationPayload.mockReturnValueOnce({
|
||||
project_id: 1,
|
||||
frame_id: 10,
|
||||
template_id: 2,
|
||||
mask_data: { polygons: [], label: '胆囊' },
|
||||
});
|
||||
apiMock.updateAnnotation.mockResolvedValueOnce({ id: 99 });
|
||||
|
||||
render(<VideoWorkspace />);
|
||||
await waitFor(() => expect(useStore.getState().frames).toHaveLength(1));
|
||||
act(() => {
|
||||
useStore.setState({
|
||||
activeTemplateId: '2',
|
||||
masks: [{
|
||||
id: 'annotation-99',
|
||||
annotationId: '99',
|
||||
frameId: '10',
|
||||
pathData: 'M 0 0 Z',
|
||||
label: '胆囊',
|
||||
color: '#ff0000',
|
||||
saveStatus: 'dirty',
|
||||
segmentation: [[0, 0, 10, 0, 10, 10]],
|
||||
bbox: [0, 0, 10, 10],
|
||||
}],
|
||||
});
|
||||
});
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '结构化归档保存' }));
|
||||
|
||||
await waitFor(() => expect(apiMock.updateAnnotation).toHaveBeenCalledWith('99', {
|
||||
template_id: 2,
|
||||
mask_data: { polygons: [], label: '胆囊' },
|
||||
points: undefined,
|
||||
bbox: undefined,
|
||||
}));
|
||||
expect(apiMock.saveAnnotation).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('deletes saved annotations when clearing current-frame masks', async () => {
|
||||
apiMock.getProjectFrames.mockResolvedValueOnce([
|
||||
{ id: 10, project_id: 1, frame_index: 0, image_url: '/frame.jpg', width: 640, height: 360 },
|
||||
]);
|
||||
apiMock.deleteAnnotation.mockResolvedValueOnce(undefined);
|
||||
|
||||
render(<VideoWorkspace />);
|
||||
await waitFor(() => expect(useStore.getState().frames).toHaveLength(1));
|
||||
act(() => {
|
||||
useStore.setState({
|
||||
masks: [
|
||||
{
|
||||
id: 'annotation-99',
|
||||
annotationId: '99',
|
||||
frameId: '10',
|
||||
pathData: 'M 0 0 Z',
|
||||
label: 'Saved',
|
||||
color: '#06b6d4',
|
||||
saved: true,
|
||||
saveStatus: 'saved',
|
||||
},
|
||||
{
|
||||
id: 'draft-1',
|
||||
frameId: '10',
|
||||
pathData: 'M 1 1 Z',
|
||||
label: 'Draft',
|
||||
color: '#ff0000',
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '清空遮罩' }));
|
||||
|
||||
await waitFor(() => expect(apiMock.deleteAnnotation).toHaveBeenCalledWith('99'));
|
||||
expect(useStore.getState().masks).toEqual([]);
|
||||
});
|
||||
|
||||
it('auto-saves pending masks before exporting COCO', async () => {
|
||||
apiMock.getProjectFrames.mockResolvedValueOnce([
|
||||
{ id: 10, project_id: 1, frame_index: 0, image_url: '/frame.jpg', width: 640, height: 360 },
|
||||
]);
|
||||
apiMock.buildAnnotationPayload.mockReturnValueOnce({ project_id: 1, frame_id: 10, mask_data: { polygons: [] } });
|
||||
apiMock.saveAnnotation.mockResolvedValueOnce({ id: 5 });
|
||||
apiMock.exportCoco.mockResolvedValueOnce(new Blob(['{}'], { type: 'application/json' }));
|
||||
|
||||
render(<VideoWorkspace />);
|
||||
await waitFor(() => expect(useStore.getState().frames).toHaveLength(1));
|
||||
act(() => {
|
||||
useStore.setState({
|
||||
masks: [{
|
||||
id: 'mask-1',
|
||||
frameId: '10',
|
||||
pathData: 'M 0 0 Z',
|
||||
label: 'AI Mask',
|
||||
color: '#06b6d4',
|
||||
segmentation: [[0, 0, 10, 0, 10, 10]],
|
||||
}],
|
||||
});
|
||||
});
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '导出 JSON 标注集' }));
|
||||
|
||||
await waitFor(() => expect(apiMock.saveAnnotation).toHaveBeenCalled());
|
||||
expect(apiMock.exportCoco).toHaveBeenCalledWith('1');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user