perf(explore): sampled, parallel visual clustering for large libraries

Computing visual clusters was O(n·k·dim) per Lloyd iteration over the whole
library, single-threaded — several seconds on an 80k-image library on first
view. Find centroids on a deterministic, evenly-strided sample (<=3000
embeddings) and then assign every image to its nearest centroid in one parallel
rayon pass. Libraries at or below the sample cap are unchanged.

Replace the greedy farthest-point seeding (which seeds outliers, leaving the
dense core under-represented on a sample so one centroid absorbed tens of
thousands of generic images) with proper density-aware k-means++ D² seeding,
made deterministic via a small fixed-seed SplitMix64 PRNG. This keeps clusters
balanced on large libraries.

A CLUSTER_CACHE_VERSION is folded into the tag-cloud cache key so existing
caches computed by the old algorithm are invalidated and recomputed. The
clustering timing line and the cache-write failure now go through the `log`
facade (debug/warn) instead of eprintln.
This commit is contained in:
2026-06-30 06:58:29 +01:00
parent 68a9df5ab3
commit d2af84d9e8
3 changed files with 154 additions and 23 deletions
+5
View File
@@ -70,6 +70,11 @@ aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
is now near-instant. The cluster cache is validated from a lightweight
image-ID signature instead of re-reading every embedding, so big libraries no
longer stall for several seconds even when the cached result is reused.
- **Faster first-time visual clustering** — computing clusters for a large
library is now much quicker: centroids are found on a representative sample and
every image is then assigned across all CPU cores, instead of iterating over
the whole library single-threaded. Density-aware seeding keeps the groupings
balanced, so no single cluster swallows a huge share of images.
- **Tag manager search and sort** — the Manage mode tag list now has a live
filter input and a sort dropdown (most-used / least-used / AZ / ZA). The
list is virtualised so libraries with thousands of tags scroll without lag,