新增 A/V 漂移診斷並釐清編碼輸入佇列語意
Docs/ChangeHistory.mdDocs/av-pipeline.mdDocs/development/AV_LAG.mdSharedCapture/StreamDiagnostics.swiftliveAPP/Capture/AVLagDiagnostics.swiftliveAPP/Capture/AVLagReportView.swiftliveAPP/StreamDiagnosticsView.swiftliveAPPTests/AVLagDiagnosticsTests.swift
commit 94669ee60f9204b6b1e9f101288a81d7673859e8Author: Nuclear0709 <agp05432@gmail.com>Date: Thu Nov 12 08:41:19 2026 +0800 新增 A/V 漂移診斷並釐清編碼輸入佇列語意 diff --git a/Docs/ChangeHistory.md b/Docs/ChangeHistory.md--- a/Docs/ChangeHistory.md+++ b/Docs/ChangeHistory.md@@ -2,6 +2,8 @@ ## 2026.11.10 音訊主時鐘與 RTMP 診斷可讀性 +- A/V Lag 新增 `sourcePTSDeltaDrift`／`sourcePTSDeltaPerMinute`／`avClockDrift`／`avSyncState`，以第一筆有效來源差為基準區分 stable／drifting／jumped；報告頁與 JSON 都能看到漂移趨勢，不直接用 `publish throughput avOffset` 判斷口型同步。+- 編碼輸入緩衝 UI 改稱「佇列保留」，新增 `received`／`consumed` 累計交接數；文件明確說明直接 handoff 時 `bytes=0`、`peakBytes=0` 是正常，不代表影格沒有進入編碼，也不代表沒有配置容量。 - TTS 最小朗讀字數預設改為 5，並把 UI 的 `@AppStorage` 與 `TTSService` 的 fallback 統一由 `TTSDefaults` 提供，避免兩邊預設值不同。最小／最大字數、佇列上限與溢出策略現在變更時立即 `updateDefault()`，不再只依賴離開設定頁或按「套用設定」才生效；`[TTS]` 同步日誌會輸出目前生效的 `minLen`／`maxLen`／`maxQueue`。 - 贊助訊息 `text` 內的圖片 URL 不再只依副檔名判斷：`extractAllImageURLs` 可接受呼叫端提供的 `knownImageURLs`（例如 `iconURL`），因此 `/media/<hash>` 這類沒有 `.png` 的網址也能被解析成行內 emoji，不再把原始 URL 留在文案中被切成多頁。插圖位置改以 UTF-16 index 計算，修正 emoji 在前時與 CoreText／NSString 位置不一致；一般非圖片連結仍保留原文。 - 音訊 PTS 缺口改為大小分流：小於 250 ms 才補零；超過時重建來源時間軸（`rebase`），不再用數秒靜音塞滿環形緩衝並擠掉歷史音訊。`AudioRingBuffer` 產生最多 8 筆 `GapEvent`，含前後 PTS、gap samples、mode、緩衝前後量與 dropped samples；SCK／ReplayKit 會寫出 `[CaptureAudioGap]` event-level log，週期摘要也新增 `zeroFill`／`rebase` 計數。diff --git a/Docs/av-pipeline.md b/Docs/av-pipeline.md--- a/Docs/av-pipeline.md+++ b/Docs/av-pipeline.md@@ -736,6 +736,8 @@ App 目前分開設定： 設定在 `screenCaptureKitVideoBufferMB`／`screenCaptureKitVideoBufferDurationMS`、`replayKitVideoBufferMB`／`replayKitVideoBufferDurationMS`。診斷會顯示 `byteLimit`、`bytes`、`queued`、`peakBytes`、`oldestAge`、capacity／expired／oversized drops、input／output idle；若 `capacityDrops` 與 `queued ≈ limit` 增加才考慮放寬，若 `expiredDrops` 增加則應先查 encoder 或 worker 效率。 +`bytes` 與 `queued` 是**取樣當下佇列仍保留的量**，不是累計送入量。`AdaptiveMediaQueue.offer()` 若發現 consumer 正在等待，會直接交接影格，不會 append 到 entries，所以 `queued=0`、`bytes=0`、甚至 `peakBytes=0` 可以是正常的即時 handoff；此時 `received`／`consumed` 仍會增加。只有在 consumer 落後時，`bytes`、`oldestAge` 與 drops 才會反映真正的 backlog。+ ### 音訊 PTS 缺口（2026-11-11） 來源 PTS 缺口不再一律補零：diff --git a/Docs/development/AV_LAG.md b/Docs/development/AV_LAG.md--- a/Docs/development/AV_LAG.md+++ b/Docs/development/AV_LAG.md@@ -27,6 +27,10 @@ videoContentAge = arrival - videoPTS | `videoSourceAge` | 加工完成時間 - 影片來源 PTS | | `audioContentAge` | 系統音訊回呼到達時間 - 音訊來源 PTS | | `sourcePTSDelta` | `videoPTS - audioPTS`，只作交叉比對 |+| `sourcePTSDeltaDrift` | 相對本 session 第一筆有效 `sourcePTSDelta` 的累積變化；負值代表音訊 PTS 持續往後 |+| `sourcePTSDeltaPerMinute` | 上述漂移換算成每分鐘變化 |+| `avClockDrift` | `audioContentAge - videoContentAge`，觀察來源時鐘相對 host clock 的漂移 |+| `avSyncState` | `stable`／`drifting`／`jumped`；只作診斷，不直接修改時間戳 | | `contentAgeJump` | 與上一次內容年齡相差 ≥1 秒時記錄；沒有階梯事件時顯示 none，不是尚未實作 | | `videoQueue` | `CaptureMailbox` 的 queued／dropped／bytes 摘要 | @@ -62,5 +66,6 @@ videoContentAge = arrival - videoPTS 1. `videoContentAge` 與 `videoQueueOldestAge` 同時階梯上升 → 影片管線 backlog，優先查 queue／PiP／主執行緒 stall。 2. `videoContentAge` 穩定但 `publishPTS - sourcePTS` 變大 → timestamp 重基準。-3. `videoSourcePTS - audioSourcePTS` 長期漂移 → 來源時鐘或 audio preferredDelta。-4. 只有 PiP 計時器落後、遊戲畫面正常 → 檢查 overlay timer 本身，不要用 timer retime 整段 video。+3. `sourcePTSDeltaDrift`／`sourcePTSDeltaPerMinute` 長期單向變化 → 來源音訊時鐘或 audio preferredDelta；先看 `avClockDrift` 是否同時變大。+4. `avSyncState=jumped` 表示 10 秒內來源 A/V 差突然超過 100 ms；只當事件，不要直接推論整段漂移。+5. 只有 PiP 計時器落後、遊戲畫面正常 → 檢查 overlay timer 本身，不要用 timer retime 整段 video。diff --git a/SharedCapture/StreamDiagnostics.swift b/SharedCapture/StreamDiagnostics.swift--- a/SharedCapture/StreamDiagnostics.swift+++ b/SharedCapture/StreamDiagnostics.swift@@ -39,6 +39,10 @@ struct StreamDiagnosticsSnapshot: Codable, Sendable { struct EncoderInputBufferDiagnostics: Codable, Sendable { let availability: String let generation: UInt64?+ /// offer() 嘗試送入的累計影格數；包含拒收，不等於已編碼數。+ let received: Int?+ /// 已交付消費端的累計影格數。+ let consumed: Int? let queued: Int? let bytes: Int? let byteLimit: Int?@@ -58,6 +62,8 @@ struct StreamDiagnosticsSnapshot: Codable, Sendable { return Self( availability: stage.availability.rawValue, generation: stage.generation,+ received: queue?.received,+ consumed: queue?.consumed, queued: queue?.queued, bytes: queue?.bytes, byteLimit: queue?.byteLimit,diff --git a/liveAPP/Capture/AVLagDiagnostics.swift b/liveAPP/Capture/AVLagDiagnostics.swift--- a/liveAPP/Capture/AVLagDiagnostics.swift+++ b/liveAPP/Capture/AVLagDiagnostics.swift@@ -14,6 +14,14 @@ struct AVLagObservation: Codable, Equatable, Sendable { var videoProcessAge: Double? var videoSourceAge: Double? var sourcePTSDelta: Double?+ /// 相對本 session 第一筆有效 `sourcePTSDelta` 的累積變化；負值代表音訊 PTS 相對影片持續往後。+ var sourcePTSDeltaDrift: Double?+ /// 上述漂移換算成每分鐘變化；取樣時間太短時為 nil。+ var sourcePTSDeltaPerMinute: Double?+ /// 音訊 host-minus-PTS 減影片 host-minus-PTS；用來觀察來源時鐘漂移。+ var avClockDrift: Double?+ /// stable / drifting / jumped；只是診斷，不直接修正時間戳。+ var avSyncState: String? var videoQueue: String? var contentAgeJump: Double? var videoFrames = 0@@ -29,6 +37,8 @@ struct AVLagObservation: Codable, Equatable, Sendable { + "videoQueueAge=\(number(videoQueueAge)) videoQueueOldestAge=\(number(videoQueueOldestAge)) " + "videoProcessAge=\(number(videoProcessAge)) " + "videoSourceAge=\(number(videoSourceAge)) sourcePTSDelta=\(number(sourcePTSDelta)) "+ + "deltaDrift=\(number(sourcePTSDeltaDrift)) deltaPerMin=\(number(sourcePTSDeltaPerMinute)) "+ + "clockDrift=\(number(avClockDrift)) syncState=\(avSyncState ?? "unknown") " + "jump=\(number(contentAgeJump)) frames=\(videoFrames)/\(audioFrames) " + "queue=\(videoQueue ?? "unknown")" }@@ -51,6 +61,10 @@ final class AVLagDiagnostics: @unchecked Sendable { private var videoFrames = 0 private var audioFrames = 0 private var lastVideoAt: Double?+ private var firstSourceDelta: Double?+ private var firstSourceDeltaAt: Double?+ private var lastSourceDelta: Double?+ private var lastSourceDeltaAt: Double? /// 影片從 SCStream 交給 worker 時呼叫；contentAge 是回呼到達時間減來源 PTS。 func recordVideo(arrival: Double, pts: Double, at now: Double,@@ -102,6 +116,44 @@ final class AVLagDiagnostics: @unchecked Sendable { } else { sourceDelta = nil }++ if let sourceDelta {+ if firstSourceDelta == nil {+ firstSourceDelta = sourceDelta+ firstSourceDeltaAt = now+ }+ }+ let sourceDeltaDrift = sourceDelta.flatMap { current in+ firstSourceDelta.map { current - $0 }+ }+ let sourceDeltaPerMinute: Double?+ if let sourceDeltaDrift, let firstSourceDeltaAt, now > firstSourceDeltaAt {+ sourceDeltaPerMinute = sourceDeltaDrift / ((now - firstSourceDeltaAt) / 60)+ } else {+ sourceDeltaPerMinute = nil+ }+ let clockDrift: Double?+ if let audioContentAge, let videoContentAge {+ clockDrift = audioContentAge - videoContentAge+ } else {+ clockDrift = nil+ }+ var syncState: String?+ if let sourceDelta {+ // 100 ms 以上的變化只在短時間內視為跳變；累積 60 秒的變化+ // 應標成 drifting，不要讓一次正常取樣窗口掩蓋漂移趨勢。+ let changedQuickly = lastSourceDeltaAt.map { now - $0 < 10 } ?? false+ let jumped = changedQuickly+ && (lastSourceDelta.map { abs(sourceDelta - $0) >= 0.100 } ?? false)+ let drifting = abs(sourceDeltaDrift ?? 0) >= 0.080+ || abs(clockDrift ?? 0) >= 0.080+ syncState = jumped ? "jumped" : (drifting ? "drifting" : "stable")+ }+ if sourceDelta != nil {+ lastSourceDelta = sourceDelta+ lastSourceDeltaAt = now+ }+ let observation = AVLagObservation( session: session, sampledAt: sampledAt,@@ -115,6 +167,10 @@ final class AVLagDiagnostics: @unchecked Sendable { videoProcessAge: videoProcessAge, videoSourceAge: videoSourceAge, sourcePTSDelta: sourceDelta,+ sourcePTSDeltaDrift: sourceDeltaDrift,+ sourcePTSDeltaPerMinute: sourceDeltaPerMinute,+ avClockDrift: clockDrift,+ avSyncState: syncState, videoQueue: videoQueue, contentAgeJump: pendingJump, videoFrames: videoFrames,diff --git a/liveAPP/Capture/AVLagReportView.swift b/liveAPP/Capture/AVLagReportView.swift--- a/liveAPP/Capture/AVLagReportView.swift+++ b/liveAPP/Capture/AVLagReportView.swift@@ -24,6 +24,10 @@ struct AVLagReportView: View { row("video process age", seconds(latest.videoProcessAge)) row("video source age", seconds(latest.videoSourceAge)) row("video PTS - audio PTS", seconds(latest.sourcePTSDelta))+ row("A/V source drift", seconds(latest.sourcePTSDeltaDrift))+ row("A/V drift per minute", seconds(latest.sourcePTSDeltaPerMinute))+ row("audio - video host offset", seconds(latest.avClockDrift))+ row("A/V sync state", latest.avSyncState ?? "unknown") row("content age jump", latest.contentAgeJump.map { seconds($0) } ?? AppLanguage.localized("avLag.noJump")) row("frames", "\(latest.videoFrames) video / \(latest.audioFrames) audio")diff --git a/liveAPP/StreamDiagnosticsView.swift b/liveAPP/StreamDiagnosticsView.swift--- a/liveAPP/StreamDiagnosticsView.swift+++ b/liveAPP/StreamDiagnosticsView.swift@@ -234,15 +234,19 @@ struct StreamDiagnosticsSections: View { group("編碼輸入緩衝") { if let input = value.encoderInput { detail("佇列狀態", input.availability)- detail("容量", [+ detail("佇列上限", [ megabytes(input.byteLimit).map { "上限 \($0)" }, input.manualFrameLimit.map { "手動上限 \($0) frames" } ?? "自動（無手動 frame cap）", input.maxAge.map { "最長 \(milliseconds($0))" } ].compactMap { $0 }.joined(separator: " · "))- detail("目前使用", [- megabytes(input.bytes).map { "已用 \($0)" },+ detail("佇列保留", [+ megabytes(input.bytes).map { "保留 \($0)" }, input.queued.map { "\($0) frames" },- megabytes(input.peakBytes).map { "峰值 \($0)" }+ megabytes(input.peakBytes).map { "佇列峰值 \($0)" }+ ].compactMap { $0 }.joined(separator: " · "))+ detail("累計交接", [+ input.received.map { "已接收 \($0)" },+ input.consumed.map { "已取出 \($0)" } ].compactMap { $0 }.joined(separator: " · ")) detail("最舊影格", input.oldestAge.map { milliseconds($0) }) detail("丟棄", [@@ -254,7 +258,7 @@ struct StreamDiagnosticsSections: View { input.inputIdle.map { "input \(milliseconds($0))" }, input.outputIdle.map { "output \(milliseconds($0))" } ].compactMap { $0 }.joined(separator: " · "))- Text("這是編碼前的原始影格 queue，不是 VT pending，也不是 RTMP 已編碼待送 bytes。")+ Text("這是編碼前的原始影格 queue，不是 VT pending，也不是 RTMP 已編碼待送 bytes。保留 0.0 MB／0 frames 可能代表 worker 正在等待、影格已直接交接；不代表沒有影格進入編碼。佇列峰值只計算實際堆積，不包含直接交接。") .font(.caption).foregroundStyle(.secondary) } else { Text("此快照沒有編碼輸入緩衝資料。")diff --git a/liveAPPTests/AVLagDiagnosticsTests.swift b/liveAPPTests/AVLagDiagnosticsTests.swift--- a/liveAPPTests/AVLagDiagnosticsTests.swift+++ b/liveAPPTests/AVLagDiagnosticsTests.swift@@ -36,6 +36,23 @@ struct AVLagDiagnosticsTests { #expect(diagnostics.snapshot(session: "s", now: 21)?.contentAgeJump == nil) } + @Test func reportsSourceDeltaDriftAndSyncState() {+ let diagnostics = AVLagDiagnostics()+ diagnostics.recordVideo(arrival: 100, pts: 100, at: 100, queue: "q", queueOldestAge: nil)+ diagnostics.recordAudio(arrival: 100, pts: 99.97, at: 100, isMic: false)+ let baseline = diagnostics.snapshot(session: "s", now: 100)+ #expect(abs((baseline?.sourcePTSDeltaDrift ?? 1) - 0) < 0.0001)+ #expect(baseline?.avSyncState == "stable")++ diagnostics.recordVideo(arrival: 160, pts: 160, at: 160, queue: "q", queueOldestAge: nil)+ diagnostics.recordAudio(arrival: 160, pts: 159.77, at: 160, isMic: false)+ let drifted = diagnostics.snapshot(session: "s", now: 160)+ #expect(abs((drifted?.sourcePTSDeltaDrift ?? 0) - 0.2) < 0.0001)+ #expect(abs((drifted?.sourcePTSDeltaPerMinute ?? 0) - 0.2) < 0.0001)+ #expect(abs((drifted?.avClockDrift ?? 0) - 0.23) < 0.0001)+ #expect(drifted?.avSyncState == "drifting")+ }+ @Test func noVideoMeansUnknownInsteadOfZero() { let diagnostics = AVLagDiagnostics() diagnostics.recordAudio(arrival: 1, pts: 1, at: 1, isMic: false)