Send speech audio as normalized PCM frames

- Resample microphone input from the actual browser AudioContext sample rate to 16k before sending it to the speech proxy.

- Encode speech input as 16-bit PCM and send it in small 1280-byte frames instead of relying on the browser to create a 16k audio context.

- Flush pending audio before sending the standard Xunfei IAT end frame.

- Extract PCM helpers and cover downsampling, PCM encoding, base64 conversion, and byte concatenation with unit tests.

- Update report editor, testing, and progress documentation for the corrected speech audio pipeline.
This commit is contained in:
2026-05-02 06:30:56 +08:00
parent 87ab7d4b9c
commit 13d8853532
6 changed files with 128 additions and 26 deletions

View File

@@ -86,7 +86,7 @@ AI 面板支持两种模式:
讯飞听写通过后端 WebSocket 代理:
- 前端连接 `/api/speech/iat`,不再生成讯飞鉴权 URL也不读取 APPID/APIKey/APISecret。
- 浏览器采集麦克风音频,转换为 16k PCM 后发送音频帧。
- 浏览器采集麦克风音频后按实际 `AudioContext.sampleRate` 重采样为 16k、16bit、单声道 PCM并按小帧发送音频帧。
- 启动前会检查浏览器是否支持 `navigator.mediaDevices.getUserMedia``AudioContext`;如果不是 `localhost` 或 HTTPS 等安全上下文浏览器会禁止麦克风能力。Docker 演示环境可使用 `https://localhost:4443`,局域网普通 HTTP 只能通过 Chrome/Edge 演示启动参数临时标记为可信来源。
- 后端读取 Settings API 中的 `xfSpeechConfig`,连接讯飞 IAT上游首帧由后端补齐 `common.app_id` 和默认 `business` 参数。
- 识别结果由后端转发回前端,并追加到 AI 输入框。