feat(discovery): AI tagging, semantic search, similarity, duplicate finder, folder management
Adds a full discovery and organisation layer to the gallery.
## AI Tagger
- WD tagger (ONNX via ort) with DirectML/CPU acceleration
- Per-image and per-folder tag queuing; configurable batch size and
confidence threshold; model downloaded on first use via ureq/zip
- Tags stored with source ('ai' | 'user'); user tags are never
overwritten by AI; AI tags protected from accidental promotion
- Tagger pause/resume per folder; in-flight batches discarded cleanly
on pause or cancellation without leaving jobs stuck in processing
## Semantic & Tag Search
- CLIP text-query embedding via candle (HuggingFace hub model)
- Progressive candidate doubling with filter post-processing so
folder/rating/media-kind filters do not silently under-return results
- Tag search with DB-backed pagination (offset + COUNT total)
- Filename search unchanged; prefix syntax: s:, t:, f: in search bar
## Similarity Search
- HNSW in-memory index (hnsw_rs) with monotonic embedding_revision
counter for cache invalidation; build_index retries if a concurrent
write advances the revision during construction
- Folder-scoped similar search uses brute-force cosine scan (no k limit)
- Region-based similarity: crop an arbitrary rectangle in the lightbox,
embed it with CLIP, find the nearest images globally or within folder
- Pagination for both similar and region results; scope toggle
(all media / current folder) re-runs the query without reopening image
## Duplicate Finder
- Three-phase detection: stat (size) → sample hash (4×16 KB windows)
→ full-file hash (xxh3, large files only) to eliminate false positives
- Filesystem-first deletion: only removes DB rows for files successfully
deleted from disk; failed files remain visible for retry
- Persisted scan cache (SQLite) with invalidation on reindex and deletion;
both global and per-folder scopes invalidated after any deletion
## Tag Cloud & Explore
- Visual cluster explore: k-means over CLIP embeddings, configurable k,
representative thumbnail per cluster; cache keyed on xxh3 of embedding set
- Tag explore: ranked tag list with image counts and representative
thumbnail per tag; invalidated when AI tagging completes or folder removed
- Tag autocomplete for search bar
## Folder Management
- Inline rename (display name only, not OS rename)
- Relocate: file-picker to update folder path; all image paths rewritten
using SUBSTR prefix replacement to avoid corrupting paths that contain
the folder name as a substring
- Missing-folder recovery banner: Locate or Remove, never silent deletion
- Right-click context menu on sidebar items: Reindex, Rename, Locate,
Remove; hover buttons preserved alongside context menu
## Background Tasks
- Per-folder progress panel with embedding, tagging, caption, and
thumbnail stage breakdown
- Retry button per task for failed embeddings and tagging jobs
- Completed-task notifications (system tray via tauri-plugin-notification)
- Scan errors shown inline; WalkDir permission errors protect existing
records from deletion rather than cascading data loss
## Backend correctness
- sqlite-vec DML performed outside transactions throughout (virtual-table
limitation); embedding revision incremented on delete as well as insert
- Tagging job discard checks status = 'processing' so paused jobs reset
to 'pending' are also skipped, not just cancelled ones
- Stale async responses in Lightbox guarded with cancellation flags and
currentImageIdRef for both tag-load and tag-add callbacks
- Duplicate cache cleared on reindex; folder-removal clears vector rows
outside transaction before deleting the folder row
This commit was merged in pull request #8.
This commit is contained in:
@@ -1,116 +1,58 @@
|
||||
# Phokus
|
||||
|
||||
## Overview
|
||||
A local-first desktop media library for browsing, filtering, and curating image and video folders.
|
||||
|
||||
Phokus is a Tauri desktop app for building a fast, local media library from folders on disk. It indexes images and videos, stores metadata in SQLite, and gives you a dense browsing workflow with filtering, favorites, ratings, and a lightbox preview.
|
||||
## Features
|
||||
|
||||
The current app is optimized for:
|
||||
|
||||
- local folders instead of cloud import flows
|
||||
- large visual libraries
|
||||
- quick review and curation
|
||||
- mixed image and video browsing
|
||||
|
||||
## Current features
|
||||
|
||||
- Add and remove media folders
|
||||
- Background indexing with progress updates
|
||||
- Browse all media or filter by folder
|
||||
- Search by filename
|
||||
- Filter by images, videos, or favorites
|
||||
- Sort by modified date, name, or file size
|
||||
- Grid density controls
|
||||
- Lightbox preview with keyboard navigation
|
||||
- Favorite and star-rating metadata saved in SQLite
|
||||
- Virtualized/local-first architecture built on Tauri + React
|
||||
- Add and remove media folders; background indexing with live progress
|
||||
- Browse all media or filter by folder, type (image/video), favorites, or star rating
|
||||
- **Filename search**, **semantic search** (`/s query`), and **tag search** (`/t tag`)
|
||||
- **Similar image search** — find visually similar media by image or selected region
|
||||
- **AI tagging** via WD tagger (ONNX, CPU/DirectML) with confidence threshold control
|
||||
- **Explore view** — visual cluster map and tag cloud for browsing by theme
|
||||
- **Duplicate finder** — scan for exact duplicates by file hash with bulk delete
|
||||
- Lightbox preview with keyboard navigation, inline tag editing, and rating controls
|
||||
- Sort by date, name, size, rating, or duration
|
||||
- Grid density controls (compact / comfortable / detail)
|
||||
|
||||
## Supported formats
|
||||
|
||||
Images:
|
||||
|
||||
- `jpg`
|
||||
- `jpeg`
|
||||
- `png`
|
||||
- `gif`
|
||||
- `bmp`
|
||||
- `tiff`
|
||||
- `tif`
|
||||
- `webp`
|
||||
- `avif`
|
||||
- `heic`
|
||||
- `heif`
|
||||
|
||||
Videos:
|
||||
|
||||
- `mp4`
|
||||
- `mov`
|
||||
- `m4v`
|
||||
- `webm`
|
||||
| Images | Videos |
|
||||
|--------|--------|
|
||||
| jpg, jpeg, png, gif, bmp | mp4, mov, m4v |
|
||||
| tiff, tif, webp, avif, heic, heif | webm |
|
||||
|
||||
## Stack
|
||||
|
||||
- Tauri 2
|
||||
- React 19
|
||||
- TypeScript
|
||||
- Zustand
|
||||
- Rust
|
||||
- SQLite + `sqlite-vec`
|
||||
- Vite
|
||||
|
||||
## Project structure
|
||||
|
||||
- `src/`: React UI, state, and components
|
||||
- `src-tauri/src/commands.rs`: Tauri command surface
|
||||
- `src-tauri/src/db.rs`: SQLite schema and queries
|
||||
- `src-tauri/src/indexer.rs`: folder crawling and batch indexing
|
||||
- `src-tauri/src/vector.rs`: vector table setup for future semantic workflows
|
||||
- Tauri 2 + Rust backend
|
||||
- React 19 + TypeScript + Zustand
|
||||
- SQLite + `sqlite-vec` (vector search)
|
||||
- ONNX Runtime (`ort`) for AI tagging
|
||||
- Candle (Rust ML) for visual embeddings
|
||||
- FFmpeg sidecar for video thumbnails and metadata
|
||||
- Vite + Tailwind CSS v4
|
||||
|
||||
## Development
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js 20+
|
||||
- `pnpm`
|
||||
- Rust toolchain
|
||||
- Tauri system prerequisites for Windows
|
||||
|
||||
### Install
|
||||
**Prerequisites:** Node.js 20+, pnpm, Rust toolchain, Tauri system prerequisites for Windows.
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
```
|
||||
|
||||
### Run in development
|
||||
# Run with hot-reload (frontend + Rust)
|
||||
pnpm dev:app
|
||||
|
||||
```bash
|
||||
pnpm tauri dev
|
||||
```
|
||||
# Frontend only
|
||||
pnpm dev:vite
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
pnpm tauri build
|
||||
# Production build
|
||||
pnpm build:app
|
||||
```
|
||||
|
||||
## How it works
|
||||
|
||||
1. Add a folder from the sidebar or Library menu.
|
||||
2. The Rust indexer walks the directory recursively.
|
||||
3. Supported files are written into SQLite with metadata such as path, size, dimensions, media type, rating, and favorite state.
|
||||
1. Add a folder from the sidebar — the Rust indexer walks it recursively.
|
||||
2. Supported files are written to SQLite with metadata (path, dimensions, media type, etc.).
|
||||
3. Background workers generate thumbnails, compute visual embeddings, and run AI tagging.
|
||||
4. Progress events stream back to the UI while the gallery updates incrementally.
|
||||
5. The gallery view loads media in pages and opens items in a lightbox for review.
|
||||
|
||||
## Notes
|
||||
|
||||
- This is currently a local desktop library, not a sync product.
|
||||
- Search is filename-based right now.
|
||||
- The vector table and embedding fields exist, but semantic search is not wired into the UI yet.
|
||||
- Some visible UI copy may still use the old working name until the frontend text is updated.
|
||||
|
||||
## Positioning
|
||||
|
||||
The clearest product description today is:
|
||||
|
||||
> A local-first desktop media library for browsing, filtering, and curating image and video folders.
|
||||
|
||||
That description is more accurate than "gallery" alone and gives you a better base for future branding, onboarding copy, and a landing page.
|
||||
5. Embeddings power semantic search and the similar images feature via an HNSW index.
|
||||
|
||||
Reference in New Issue
Block a user