Files

T

LyAhn 0ca4d142d8 feat(discovery): AI tagging, semantic search, similarity, duplicate finder, folder management

Adds a full discovery and organisation layer to the gallery.

## AI Tagger
- WD tagger (ONNX via ort) with DirectML/CPU acceleration
- Per-image and per-folder tag queuing; configurable batch size and
  confidence threshold; model downloaded on first use via ureq/zip
- Tags stored with source ('ai' | 'user'); user tags are never
  overwritten by AI; AI tags protected from accidental promotion
- Tagger pause/resume per folder; in-flight batches discarded cleanly
  on pause or cancellation without leaving jobs stuck in processing

## Semantic & Tag Search
- CLIP text-query embedding via candle (HuggingFace hub model)
- Progressive candidate doubling with filter post-processing so
  folder/rating/media-kind filters do not silently under-return results
- Tag search with DB-backed pagination (offset + COUNT total)
- Filename search unchanged; prefix syntax: s:, t:, f: in search bar

## Similarity Search
- HNSW in-memory index (hnsw_rs) with monotonic embedding_revision
  counter for cache invalidation; build_index retries if a concurrent
  write advances the revision during construction
- Folder-scoped similar search uses brute-force cosine scan (no k limit)
- Region-based similarity: crop an arbitrary rectangle in the lightbox,
  embed it with CLIP, find the nearest images globally or within folder
- Pagination for both similar and region results; scope toggle
  (all media / current folder) re-runs the query without reopening image

## Duplicate Finder
- Three-phase detection: stat (size) → sample hash (4×16 KB windows)
  → full-file hash (xxh3, large files only) to eliminate false positives
- Filesystem-first deletion: only removes DB rows for files successfully
  deleted from disk; failed files remain visible for retry
- Persisted scan cache (SQLite) with invalidation on reindex and deletion;
  both global and per-folder scopes invalidated after any deletion

## Tag Cloud & Explore
- Visual cluster explore: k-means over CLIP embeddings, configurable k,
  representative thumbnail per cluster; cache keyed on xxh3 of embedding set
- Tag explore: ranked tag list with image counts and representative
  thumbnail per tag; invalidated when AI tagging completes or folder removed
- Tag autocomplete for search bar

## Folder Management
- Inline rename (display name only, not OS rename)
- Relocate: file-picker to update folder path; all image paths rewritten
  using SUBSTR prefix replacement to avoid corrupting paths that contain
  the folder name as a substring
- Missing-folder recovery banner: Locate or Remove, never silent deletion
- Right-click context menu on sidebar items: Reindex, Rename, Locate,
  Remove; hover buttons preserved alongside context menu

## Background Tasks
- Per-folder progress panel with embedding, tagging, caption, and
  thumbnail stage breakdown
- Retry button per task for failed embeddings and tagging jobs
- Completed-task notifications (system tray via tauri-plugin-notification)
- Scan errors shown inline; WalkDir permission errors protect existing
  records from deletion rather than cascading data loss

## Backend correctness
- sqlite-vec DML performed outside transactions throughout (virtual-table
  limitation); embedding revision incremented on delete as well as insert
- Tagging job discard checks status = 'processing' so paused jobs reset
  to 'pending' are also skipped, not just cancelled ones
- Stale async responses in Lightbox guarded with cancellation flags and
  currentImageIdRef for both tag-load and tag-add callbacks
- Duplicate cache cleared on reindex; folder-removal clears vector rows
  outside transaction before deleting the folder row

2026-06-07 22:43:16 +00:00

4.5 KiB

Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project overview

Phokus — a Tauri v2 desktop image gallery app with a React/TypeScript frontend and a Rust backend. The app indexes local media folders, generates thumbnails via FFmpeg, computes visual embeddings for semantic search and similarity, and AI-tags images using the WD tagger (ONNX via ort). AI captioning code exists in the backend but the UI surface has been removed; the worker is commented out in lib.rs.

Commands

# Development (Vite hot-reload + Rust auto-rebuild)
pnpm dev:app

# Frontend only (no Tauri window)
pnpm dev:vite

# Production build
pnpm build:app

# Type-check frontend
pnpm build:vite

Use pnpm — never npm.

There are no test suites configured.

Architecture

Frontend (`src/`)

store.ts — single Zustand store (useGalleryStore) that owns all app state and all invoke() calls to the Tauri backend. Every feature (folders, images, search, similar images, tags, captions, tagger, duplicates) is implemented as store actions here. React components are thin consumers.
App.tsx — sets up Tauri event listeners (subscribeToProgress) and renders the top-level layout (sidebar + active view).
src/components/ — UI components: Gallery, Lightbox, Sidebar, Toolbar, TagCloud, DuplicateFinder, BackgroundTasks, SettingsModal, MenuBar, TitleBar.
State management: Zustand v5, no selectors library — components call useGalleryStore(s => s.field) directly.
Styling: Tailwind CSS v4 (Vite plugin, no config file).
Virtualized gallery grid: @tanstack/react-virtual.
Animation: framer-motion.

Search modes

The search bar supports prefix syntax parsed by parseSearchValue in store.ts:

No prefix / f: — filename search (paginated, DB-backed)
/s <query> or s: <query> — semantic (embedding) search
/t <tag> or t: <tag> — tag search

Backend (`src-tauri/src/`)

Workers are started in lib.rs and run as background threads throughout the app lifetime:

thumbnail worker (multiple threads, count from StorageProfile::Balanced)
metadata worker — FFmpeg probe for video files
embedding worker — generates CLIP-style visual embeddings (candle, HuggingFace hub)
tagging worker — WD tagger via ONNX Runtime (ort), DirectML/CPU acceleration

Key modules:

File	Purpose
`db.rs`	SQLite pool (r2d2 + rusqlite), schema migrations, all query functions
`commands.rs`	All `#[tauri::command]` handlers — one-to-one with frontend `invoke()` calls
`indexer.rs`	Worker thread launchers and job dispatch
`embedder.rs`	Visual embedding generation (candle + HF hub models)
`vector.rs`	sqlite-vec integration + HNSW index for ANN search
`hnsw_index.rs`	In-memory HNSW index wrapper (hnsw_rs)
`tagger.rs`	WD tagger: ONNX model download, inference, CSV tag loading
`captioner.rs`	AI captioning (ONNX, disabled in workers but code intact)
`thumbnail.rs`	Thumbnail generation (image crate + fast_image_resize, FFmpeg for video)
`media.rs`	FFmpeg sidecar provisioning and probing
`storage.rs`	`StorageProfile` for tuning worker counts

Database: SQLite with WAL mode, stored in the Tauri app data directory as gallery.db. Thumbnails stored alongside as thumbnails/.

Tauri events (backend → frontend)

Event	Payload
`index-progress`	`IndexProgress`
`media-job-progress`	`MediaJobProgressEvent`
`indexed-images`	`IndexedImagesBatch`
`media-updated`	`ThumbnailBatch`
`caption-model-progress`	`CaptionModelProgress`
`tagger-model-progress`	`TaggerModelProgress`
`duplicate_scan_progress`	`[scanned, total]`

Key types

ImageRecord (mirrored in store.ts and db.rs) is the central data type. It carries embedding status, tagging status, caption data, and media metadata. The frontend type must stay in sync with the Rust struct serialization.

Development notes

Hot Reload is active during dev:app — do not restart the server for frontend changes. Restart only when adding Rust crates or changing Vite config.
ML inference crates (candle-*, ort, image, rayon, tokenizers, xxhash-rust, rusqlite) use opt-level = 3 in dev profile to keep inference performance acceptable.
The caption worker is intentionally disabled (lib.rs:73) — the backend code is intact for future re-enabling.
Never use any type in TypeScript — look up correct types.

4.5 KiB Raw Blame History