diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..e9febb9 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,94 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project overview + +**Phokus** — a Tauri v2 desktop image gallery app with a React/TypeScript frontend and a Rust backend. The app indexes local media folders, generates thumbnails via FFmpeg, computes visual embeddings for semantic search and similarity, and AI-tags images using the WD tagger (ONNX via `ort`). AI captioning code exists in the backend but the UI surface has been removed; the worker is commented out in `lib.rs`. + +## Commands + +```bash +# Development (Vite hot-reload + Rust auto-rebuild) +pnpm dev:app + +# Frontend only (no Tauri window) +pnpm dev:vite + +# Production build +pnpm build:app + +# Type-check frontend +pnpm build:vite +``` + +Use **pnpm** — never npm. + +There are no test suites configured. + +## Architecture + +### Frontend (`src/`) + +- **`store.ts`** — single Zustand store (`useGalleryStore`) that owns all app state and all `invoke()` calls to the Tauri backend. Every feature (folders, images, search, similar images, tags, captions, tagger, duplicates) is implemented as store actions here. React components are thin consumers. +- **`App.tsx`** — sets up Tauri event listeners (`subscribeToProgress`) and renders the top-level layout (sidebar + active view). +- **`src/components/`** — UI components: `Gallery`, `Lightbox`, `Sidebar`, `Toolbar`, `TagCloud`, `DuplicateFinder`, `BackgroundTasks`, `SettingsModal`, `MenuBar`, `TitleBar`. +- State management: Zustand v5, no selectors library — components call `useGalleryStore(s => s.field)` directly. +- Styling: Tailwind CSS v4 (Vite plugin, no config file). +- Virtualized gallery grid: `@tanstack/react-virtual`. +- Animation: `framer-motion`. + +### Search modes + +The search bar supports prefix syntax parsed by `parseSearchValue` in `store.ts`: +- No prefix / `f:` — filename search (paginated, DB-backed) +- `/s ` or `s: ` — semantic (embedding) search +- `/t ` or `t: ` — tag search + +### Backend (`src-tauri/src/`) + +Workers are started in `lib.rs` and run as background threads throughout the app lifetime: +- **thumbnail worker** (multiple threads, count from `StorageProfile::Balanced`) +- **metadata worker** — FFmpeg probe for video files +- **embedding worker** — generates CLIP-style visual embeddings (candle, HuggingFace hub) +- **tagging worker** — WD tagger via ONNX Runtime (`ort`), DirectML/CPU acceleration + +Key modules: +| File | Purpose | +|------|---------| +| `db.rs` | SQLite pool (r2d2 + rusqlite), schema migrations, all query functions | +| `commands.rs` | All `#[tauri::command]` handlers — one-to-one with frontend `invoke()` calls | +| `indexer.rs` | Worker thread launchers and job dispatch | +| `embedder.rs` | Visual embedding generation (candle + HF hub models) | +| `vector.rs` | sqlite-vec integration + HNSW index for ANN search | +| `hnsw_index.rs` | In-memory HNSW index wrapper (hnsw_rs) | +| `tagger.rs` | WD tagger: ONNX model download, inference, CSV tag loading | +| `captioner.rs` | AI captioning (ONNX, disabled in workers but code intact) | +| `thumbnail.rs` | Thumbnail generation (image crate + fast_image_resize, FFmpeg for video) | +| `media.rs` | FFmpeg sidecar provisioning and probing | +| `storage.rs` | `StorageProfile` for tuning worker counts | + +Database: SQLite with WAL mode, stored in the Tauri app data directory as `gallery.db`. Thumbnails stored alongside as `thumbnails/`. + +### Tauri events (backend → frontend) + +| Event | Payload | +|-------|---------| +| `index-progress` | `IndexProgress` | +| `media-job-progress` | `MediaJobProgressEvent` | +| `indexed-images` | `IndexedImagesBatch` | +| `media-updated` | `ThumbnailBatch` | +| `caption-model-progress` | `CaptionModelProgress` | +| `tagger-model-progress` | `TaggerModelProgress` | +| `duplicate_scan_progress` | `[scanned, total]` | + +### Key types + +`ImageRecord` (mirrored in `store.ts` and `db.rs`) is the central data type. It carries embedding status, tagging status, caption data, and media metadata. The frontend type must stay in sync with the Rust struct serialization. + +## Development notes + +- Hot Reload is active during `dev:app` — do not restart the server for frontend changes. Restart only when adding Rust crates or changing Vite config. +- ML inference crates (`candle-*`, `ort`, `image`, `rayon`, `tokenizers`, `xxhash-rust`, `rusqlite`) use `opt-level = 3` in dev profile to keep inference performance acceptable. +- The caption worker is intentionally disabled (`lib.rs:73`) — the backend code is intact for future re-enabling. +- **Never use `any` type** in TypeScript — look up correct types. diff --git a/README.md b/README.md index a4b5578..784cc6b 100644 --- a/README.md +++ b/README.md @@ -1,116 +1,58 @@ # Phokus -## Overview +A local-first desktop media library for browsing, filtering, and curating image and video folders. -Phokus is a Tauri desktop app for building a fast, local media library from folders on disk. It indexes images and videos, stores metadata in SQLite, and gives you a dense browsing workflow with filtering, favorites, ratings, and a lightbox preview. +## Features -The current app is optimized for: - -- local folders instead of cloud import flows -- large visual libraries -- quick review and curation -- mixed image and video browsing - -## Current features - -- Add and remove media folders -- Background indexing with progress updates -- Browse all media or filter by folder -- Search by filename -- Filter by images, videos, or favorites -- Sort by modified date, name, or file size -- Grid density controls -- Lightbox preview with keyboard navigation -- Favorite and star-rating metadata saved in SQLite -- Virtualized/local-first architecture built on Tauri + React +- Add and remove media folders; background indexing with live progress +- Browse all media or filter by folder, type (image/video), favorites, or star rating +- **Filename search**, **semantic search** (`/s query`), and **tag search** (`/t tag`) +- **Similar image search** — find visually similar media by image or selected region +- **AI tagging** via WD tagger (ONNX, CPU/DirectML) with confidence threshold control +- **Explore view** — visual cluster map and tag cloud for browsing by theme +- **Duplicate finder** — scan for exact duplicates by file hash with bulk delete +- Lightbox preview with keyboard navigation, inline tag editing, and rating controls +- Sort by date, name, size, rating, or duration +- Grid density controls (compact / comfortable / detail) ## Supported formats -Images: - -- `jpg` -- `jpeg` -- `png` -- `gif` -- `bmp` -- `tiff` -- `tif` -- `webp` -- `avif` -- `heic` -- `heif` - -Videos: - -- `mp4` -- `mov` -- `m4v` -- `webm` +| Images | Videos | +|--------|--------| +| jpg, jpeg, png, gif, bmp | mp4, mov, m4v | +| tiff, tif, webp, avif, heic, heif | webm | ## Stack -- Tauri 2 -- React 19 -- TypeScript -- Zustand -- Rust -- SQLite + `sqlite-vec` -- Vite - -## Project structure - -- `src/`: React UI, state, and components -- `src-tauri/src/commands.rs`: Tauri command surface -- `src-tauri/src/db.rs`: SQLite schema and queries -- `src-tauri/src/indexer.rs`: folder crawling and batch indexing -- `src-tauri/src/vector.rs`: vector table setup for future semantic workflows +- Tauri 2 + Rust backend +- React 19 + TypeScript + Zustand +- SQLite + `sqlite-vec` (vector search) +- ONNX Runtime (`ort`) for AI tagging +- Candle (Rust ML) for visual embeddings +- FFmpeg sidecar for video thumbnails and metadata +- Vite + Tailwind CSS v4 ## Development -### Prerequisites - -- Node.js 20+ -- `pnpm` -- Rust toolchain -- Tauri system prerequisites for Windows - -### Install +**Prerequisites:** Node.js 20+, pnpm, Rust toolchain, Tauri system prerequisites for Windows. ```bash pnpm install -``` -### Run in development +# Run with hot-reload (frontend + Rust) +pnpm dev:app -```bash -pnpm tauri dev -``` +# Frontend only +pnpm dev:vite -### Build - -```bash -pnpm tauri build +# Production build +pnpm build:app ``` ## How it works -1. Add a folder from the sidebar or Library menu. -2. The Rust indexer walks the directory recursively. -3. Supported files are written into SQLite with metadata such as path, size, dimensions, media type, rating, and favorite state. +1. Add a folder from the sidebar — the Rust indexer walks it recursively. +2. Supported files are written to SQLite with metadata (path, dimensions, media type, etc.). +3. Background workers generate thumbnails, compute visual embeddings, and run AI tagging. 4. Progress events stream back to the UI while the gallery updates incrementally. -5. The gallery view loads media in pages and opens items in a lightbox for review. - -## Notes - -- This is currently a local desktop library, not a sync product. -- Search is filename-based right now. -- The vector table and embedding fields exist, but semantic search is not wired into the UI yet. -- Some visible UI copy may still use the old working name until the frontend text is updated. - -## Positioning - -The clearest product description today is: - -> A local-first desktop media library for browsing, filtering, and curating image and video folders. - -That description is more accurate than "gallery" alone and gives you a better base for future branding, onboarding copy, and a landing page. +5. Embeddings power semantic search and the similar images feature via an HNSW index.