feat(discovery): AI tagging, semantic search, similarity, duplicate finder, folder management

Adds a full discovery and organisation layer to the gallery.

## AI Tagger
- WD tagger (ONNX via ort) with DirectML/CPU acceleration
- Per-image and per-folder tag queuing; configurable batch size and
  confidence threshold; model downloaded on first use via ureq/zip
- Tags stored with source ('ai' | 'user'); user tags are never
  overwritten by AI; AI tags protected from accidental promotion
- Tagger pause/resume per folder; in-flight batches discarded cleanly
  on pause or cancellation without leaving jobs stuck in processing

## Semantic & Tag Search
- CLIP text-query embedding via candle (HuggingFace hub model)
- Progressive candidate doubling with filter post-processing so
  folder/rating/media-kind filters do not silently under-return results
- Tag search with DB-backed pagination (offset + COUNT total)
- Filename search unchanged; prefix syntax: s:, t:, f: in search bar

## Similarity Search
- HNSW in-memory index (hnsw_rs) with monotonic embedding_revision
  counter for cache invalidation; build_index retries if a concurrent
  write advances the revision during construction
- Folder-scoped similar search uses brute-force cosine scan (no k limit)
- Region-based similarity: crop an arbitrary rectangle in the lightbox,
  embed it with CLIP, find the nearest images globally or within folder
- Pagination for both similar and region results; scope toggle
  (all media / current folder) re-runs the query without reopening image

## Duplicate Finder
- Three-phase detection: stat (size) → sample hash (4×16 KB windows)
  → full-file hash (xxh3, large files only) to eliminate false positives
- Filesystem-first deletion: only removes DB rows for files successfully
  deleted from disk; failed files remain visible for retry
- Persisted scan cache (SQLite) with invalidation on reindex and deletion;
  both global and per-folder scopes invalidated after any deletion

## Tag Cloud & Explore
- Visual cluster explore: k-means over CLIP embeddings, configurable k,
  representative thumbnail per cluster; cache keyed on xxh3 of embedding set
- Tag explore: ranked tag list with image counts and representative
  thumbnail per tag; invalidated when AI tagging completes or folder removed
- Tag autocomplete for search bar

## Folder Management
- Inline rename (display name only, not OS rename)
- Relocate: file-picker to update folder path; all image paths rewritten
  using SUBSTR prefix replacement to avoid corrupting paths that contain
  the folder name as a substring
- Missing-folder recovery banner: Locate or Remove, never silent deletion
- Right-click context menu on sidebar items: Reindex, Rename, Locate,
  Remove; hover buttons preserved alongside context menu

## Background Tasks
- Per-folder progress panel with embedding, tagging, caption, and
  thumbnail stage breakdown
- Retry button per task for failed embeddings and tagging jobs
- Completed-task notifications (system tray via tauri-plugin-notification)
- Scan errors shown inline; WalkDir permission errors protect existing
  records from deletion rather than cascading data loss

## Backend correctness
- sqlite-vec DML performed outside transactions throughout (virtual-table
  limitation); embedding revision incremented on delete as well as insert
- Tagging job discard checks status = 'processing' so paused jobs reset
  to 'pending' are also skipped, not just cancelled ones
- Stale async responses in Lightbox guarded with cancellation flags and
  currentImageIdRef for both tag-load and tag-add callbacks
- Duplicate cache cleared on reindex; folder-removal clears vector rows
  outside transaction before deleting the folder row
This commit was merged in pull request #8.
This commit is contained in:
2026-06-07 22:43:16 +00:00
parent 8905baf4a5
commit 0ca4d142d8
32 changed files with 9539 additions and 760 deletions
+94
View File
@@ -0,0 +1,94 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project overview
**Phokus** — a Tauri v2 desktop image gallery app with a React/TypeScript frontend and a Rust backend. The app indexes local media folders, generates thumbnails via FFmpeg, computes visual embeddings for semantic search and similarity, and AI-tags images using the WD tagger (ONNX via `ort`). AI captioning code exists in the backend but the UI surface has been removed; the worker is commented out in `lib.rs`.
## Commands
```bash
# Development (Vite hot-reload + Rust auto-rebuild)
pnpm dev:app
# Frontend only (no Tauri window)
pnpm dev:vite
# Production build
pnpm build:app
# Type-check frontend
pnpm build:vite
```
Use **pnpm** — never npm.
There are no test suites configured.
## Architecture
### Frontend (`src/`)
- **`store.ts`** — single Zustand store (`useGalleryStore`) that owns all app state and all `invoke()` calls to the Tauri backend. Every feature (folders, images, search, similar images, tags, captions, tagger, duplicates) is implemented as store actions here. React components are thin consumers.
- **`App.tsx`** — sets up Tauri event listeners (`subscribeToProgress`) and renders the top-level layout (sidebar + active view).
- **`src/components/`** — UI components: `Gallery`, `Lightbox`, `Sidebar`, `Toolbar`, `TagCloud`, `DuplicateFinder`, `BackgroundTasks`, `SettingsModal`, `MenuBar`, `TitleBar`.
- State management: Zustand v5, no selectors library — components call `useGalleryStore(s => s.field)` directly.
- Styling: Tailwind CSS v4 (Vite plugin, no config file).
- Virtualized gallery grid: `@tanstack/react-virtual`.
- Animation: `framer-motion`.
### Search modes
The search bar supports prefix syntax parsed by `parseSearchValue` in `store.ts`:
- No prefix / `f:` — filename search (paginated, DB-backed)
- `/s <query>` or `s: <query>` — semantic (embedding) search
- `/t <tag>` or `t: <tag>` — tag search
### Backend (`src-tauri/src/`)
Workers are started in `lib.rs` and run as background threads throughout the app lifetime:
- **thumbnail worker** (multiple threads, count from `StorageProfile::Balanced`)
- **metadata worker** — FFmpeg probe for video files
- **embedding worker** — generates CLIP-style visual embeddings (candle, HuggingFace hub)
- **tagging worker** — WD tagger via ONNX Runtime (`ort`), DirectML/CPU acceleration
Key modules:
| File | Purpose |
|------|---------|
| `db.rs` | SQLite pool (r2d2 + rusqlite), schema migrations, all query functions |
| `commands.rs` | All `#[tauri::command]` handlers — one-to-one with frontend `invoke()` calls |
| `indexer.rs` | Worker thread launchers and job dispatch |
| `embedder.rs` | Visual embedding generation (candle + HF hub models) |
| `vector.rs` | sqlite-vec integration + HNSW index for ANN search |
| `hnsw_index.rs` | In-memory HNSW index wrapper (hnsw_rs) |
| `tagger.rs` | WD tagger: ONNX model download, inference, CSV tag loading |
| `captioner.rs` | AI captioning (ONNX, disabled in workers but code intact) |
| `thumbnail.rs` | Thumbnail generation (image crate + fast_image_resize, FFmpeg for video) |
| `media.rs` | FFmpeg sidecar provisioning and probing |
| `storage.rs` | `StorageProfile` for tuning worker counts |
Database: SQLite with WAL mode, stored in the Tauri app data directory as `gallery.db`. Thumbnails stored alongside as `thumbnails/`.
### Tauri events (backend → frontend)
| Event | Payload |
|-------|---------|
| `index-progress` | `IndexProgress` |
| `media-job-progress` | `MediaJobProgressEvent` |
| `indexed-images` | `IndexedImagesBatch` |
| `media-updated` | `ThumbnailBatch` |
| `caption-model-progress` | `CaptionModelProgress` |
| `tagger-model-progress` | `TaggerModelProgress` |
| `duplicate_scan_progress` | `[scanned, total]` |
### Key types
`ImageRecord` (mirrored in `store.ts` and `db.rs`) is the central data type. It carries embedding status, tagging status, caption data, and media metadata. The frontend type must stay in sync with the Rust struct serialization.
## Development notes
- Hot Reload is active during `dev:app` — do not restart the server for frontend changes. Restart only when adding Rust crates or changing Vite config.
- ML inference crates (`candle-*`, `ort`, `image`, `rayon`, `tokenizers`, `xxhash-rust`, `rusqlite`) use `opt-level = 3` in dev profile to keep inference performance acceptable.
- The caption worker is intentionally disabled (`lib.rs:73`) — the backend code is intact for future re-enabling.
- **Never use `any` type** in TypeScript — look up correct types.