perf(tagger): chunked, yielding inference so tagging stops freezing the UI

The tagging worker claimed a batch from the DB but ran the model one image
at a time, so `tagger_batch_size` had no effect on inference. Batching the
whole claim into a single DirectML forward pass fixed that but introduced a
worse problem: on a shared GPU each wide dispatch locks the device (and the
WebView2 compositor with it) for 1-3.7s, freezing the entire app while
tagging runs — worst of all on first launch with a cold graph compile.

The WD model is compute-bound here (~50-230ms/image of actual GPU work), so
a wide batch buys almost no throughput; it only lengthens each uninterruptible
GPU lock. So decouple DB claim size from GPU granularity: claim
`tagger_batch_size` for DB efficiency, but feed the GPU in TAGGER_INFER_CHUNK
(4) images per forward pass with a brief yield between chunks. Each dispatch
now lasts ~200-900ms, the UI gets windows to paint, peak decode memory is
bounded, and the cold compile is for a smaller shape. As a bonus the wide-batch
throughput spikes disappear — steady ~1.6-1.8s/16 vs the old 0.8-3.7s swings.

- run_batch: parallel (rayon) decode + single forward pass per chunk, with
  per-image fallback and decode failures kept attached to their input slot.
- Worker iterates source_paths.chunks(TAGGER_INFER_CHUNK), yielding 40ms
  between chunks; outputs zip back to jobs 1:1, write tx unchanged.
- Remove now-dead WdTagger::run() (fallback uses infer_one directly).
This commit is contained in:
2026-06-29 01:09:47 +01:00
parent cebd709391
commit 992417710f
3 changed files with 200 additions and 49 deletions
+6
View File
@@ -93,6 +93,12 @@ aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
- **Explore polish** — the Tag Cloud now uses the in-app tooltip instead of the
native browser one (and reads "1 image", not "1 images"), and the folder-scope
dropdown no longer opens behind the cluster cards.
- **AI tagging no longer freezes the app** — tagging now runs inference in small
GPU micro-batches with a brief yield between them, instead of one wide batch
that monopolised the GPU (and with it the whole UI) for seconds at a time. The
app stays responsive while tagging runs, throughput is steadier (the old wide
batches caused periodic slowdowns), and the first batch after launch starts
faster.
## [0.1.1] — 2026-06-23