perf(tagger): multi-thread CPU inference instead of pinning to one core

The ONNX session was built with intra_threads(1) unconditionally. That's
correct for DirectML (compute is on the GPU), but on the CPU execution
provider it pinned all matmul/conv work to a single core — ~1.78s/image,
~14s for an 8-image batch.

Derive the intra-op thread count from available_parallelism() for the CPU EP
(leaving 2 logical cores free for the UI and a possible concurrent scan;
tagging is the lowest-priority worker, so heavier workers are idle when it
runs). DirectML/Auto keep a single thread. Measured ~2.7x on a representative
machine (14s -> 5.3s per 8-image batch); sublinear because swinv2 inference is
memory-bandwidth-bound. The selected thread count is logged at load.
This commit is contained in:
2026-06-29 09:09:29 +01:00
parent 992417710f
commit 71ad7bf762
2 changed files with 24 additions and 2 deletions
+4
View File
@@ -59,6 +59,10 @@ aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
- **Explore cluster layout** — clusters are now sized by image count (busier - **Explore cluster layout** — clusters are now sized by image count (busier
clusters are larger and stack on top) and repositioned to avoid overlapping, so clusters are larger and stack on top) and repositioned to avoid overlapping, so
every cluster stays viewable and clickable even in dense libraries. every cluster stays viewable and clickable even in dense libraries.
- **Faster CPU tagging** — when AI tagging runs on the CPU provider (no usable
GPU) it now uses multiple cores instead of being pinned to one, several times
faster on multi-core machines. A couple of cores are left free so the rest of
the app stays responsive. GPU (DirectML) tagging is unchanged.
### Fixed ### Fixed
+20 -2
View File
@@ -693,6 +693,22 @@ fn create_tagger_session(path: &Path, acceleration: TaggerAcceleration) -> Resul
TaggerAcceleration::Auto | TaggerAcceleration::Directml TaggerAcceleration::Auto | TaggerAcceleration::Directml
); );
// Intra-op thread count. For DirectML the matmuls run on the GPU, so a
// single CPU thread for the few CPU-side ops avoids needlessly contending
// with the other workers. The CPU EP, however, is compute-bound and would
// otherwise be pinned to one core (~15x slower than it needs to be), so give
// it most of the logical cores while leaving a couple free for the UI and a
// possible concurrent scan. Tagging is the lowest-priority worker, so when
// it runs the heavier workers are idle. (Auto only lands on CPU when
// DirectML is unavailable — rare on Windows — and stays single-threaded;
// CPU-bound users can select the CPU provider explicitly for the speedup.)
let intra_threads = match acceleration {
TaggerAcceleration::Cpu => std::thread::available_parallelism()
.map(|p| p.get().saturating_sub(2).max(1))
.unwrap_or(2),
TaggerAcceleration::Auto | TaggerAcceleration::Directml => 1,
};
let builder = builder let builder = builder
.with_memory_pattern(!use_directml) .with_memory_pattern(!use_directml)
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
@@ -700,12 +716,14 @@ fn create_tagger_session(path: &Path, acceleration: TaggerAcceleration) -> Resul
.with_parallel_execution(false) .with_parallel_execution(false)
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder let builder = builder
.with_intra_threads(1) .with_intra_threads(intra_threads)
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
let mut builder = match acceleration { let mut builder = match acceleration {
TaggerAcceleration::Cpu => { TaggerAcceleration::Cpu => {
log::info!("WD tagger: using CPU execution provider"); log::info!(
"WD tagger: using CPU execution provider ({intra_threads} intra-op threads)"
);
builder builder
} }
TaggerAcceleration::Auto => builder TaggerAcceleration::Auto => builder