Compare commits

...

5 Commits

Author SHA1 Message Date
LyAhn d7595703de docs: rewrite README to reflect current features, add CLAUDE.md 2026-06-02 19:40:00 +01:00
LyAhn 5b35bc5b6e chore: set app title to Phokus, remove default Vite favicon 2026-06-02 19:39:44 +01:00
LyAhn df17497808 feat: add region-based similarity search
Adds a "Search within image" button to the lightbox that lets the user
draw a crop region on an image and find visually similar results using
that crop's embedding. The crop is embedded in-memory without a temp
file via a new embed_image_crop method on ClipImageEmbedder.

Introduces a folder-scoped cosine search (search_image_ids_by_embedding_in_folder)
to support the current-folder scope option, and wires up the new
find_similar_by_region Tauri command end-to-end from Rust through to
the Zustand store and Lightbox UI.
2026-06-02 19:39:31 +01:00
LyAhn ff4a568b57 feat: expand media exploration and tagging controls 2026-04-12 12:18:47 +01:00
LyAhn b2826d1143 feat: add WD tagger with CSV tag support and model download via ureq/zip
Introduces tagger.rs for WD-based image tagging with CSV label support.
Adds csv, ureq, and zip dependencies; updates gitignore to exclude Python/JSON files.
2026-04-08 20:04:37 +01:00
29 changed files with 5935 additions and 1014 deletions
+6
View File
@@ -29,3 +29,9 @@ dist-ssr
# Local staging area
/staging
# Misc
*.py
*.json
*.pyc
+94
View File
@@ -0,0 +1,94 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project overview
**Phokus** — a Tauri v2 desktop image gallery app with a React/TypeScript frontend and a Rust backend. The app indexes local media folders, generates thumbnails via FFmpeg, computes visual embeddings for semantic search and similarity, and AI-tags images using the WD tagger (ONNX via `ort`). AI captioning code exists in the backend but the UI surface has been removed; the worker is commented out in `lib.rs`.
## Commands
```bash
# Development (Vite hot-reload + Rust auto-rebuild)
pnpm dev:app
# Frontend only (no Tauri window)
pnpm dev:vite
# Production build
pnpm build:app
# Type-check frontend
pnpm build:vite
```
Use **pnpm** — never npm.
There are no test suites configured.
## Architecture
### Frontend (`src/`)
- **`store.ts`** — single Zustand store (`useGalleryStore`) that owns all app state and all `invoke()` calls to the Tauri backend. Every feature (folders, images, search, similar images, tags, captions, tagger, duplicates) is implemented as store actions here. React components are thin consumers.
- **`App.tsx`** — sets up Tauri event listeners (`subscribeToProgress`) and renders the top-level layout (sidebar + active view).
- **`src/components/`** — UI components: `Gallery`, `Lightbox`, `Sidebar`, `Toolbar`, `TagCloud`, `DuplicateFinder`, `BackgroundTasks`, `SettingsModal`, `MenuBar`, `TitleBar`.
- State management: Zustand v5, no selectors library — components call `useGalleryStore(s => s.field)` directly.
- Styling: Tailwind CSS v4 (Vite plugin, no config file).
- Virtualized gallery grid: `@tanstack/react-virtual`.
- Animation: `framer-motion`.
### Search modes
The search bar supports prefix syntax parsed by `parseSearchValue` in `store.ts`:
- No prefix / `f:` — filename search (paginated, DB-backed)
- `/s <query>` or `s: <query>` — semantic (embedding) search
- `/t <tag>` or `t: <tag>` — tag search
### Backend (`src-tauri/src/`)
Workers are started in `lib.rs` and run as background threads throughout the app lifetime:
- **thumbnail worker** (multiple threads, count from `StorageProfile::Balanced`)
- **metadata worker** — FFmpeg probe for video files
- **embedding worker** — generates CLIP-style visual embeddings (candle, HuggingFace hub)
- **tagging worker** — WD tagger via ONNX Runtime (`ort`), DirectML/CPU acceleration
Key modules:
| File | Purpose |
|------|---------|
| `db.rs` | SQLite pool (r2d2 + rusqlite), schema migrations, all query functions |
| `commands.rs` | All `#[tauri::command]` handlers — one-to-one with frontend `invoke()` calls |
| `indexer.rs` | Worker thread launchers and job dispatch |
| `embedder.rs` | Visual embedding generation (candle + HF hub models) |
| `vector.rs` | sqlite-vec integration + HNSW index for ANN search |
| `hnsw_index.rs` | In-memory HNSW index wrapper (hnsw_rs) |
| `tagger.rs` | WD tagger: ONNX model download, inference, CSV tag loading |
| `captioner.rs` | AI captioning (ONNX, disabled in workers but code intact) |
| `thumbnail.rs` | Thumbnail generation (image crate + fast_image_resize, FFmpeg for video) |
| `media.rs` | FFmpeg sidecar provisioning and probing |
| `storage.rs` | `StorageProfile` for tuning worker counts |
Database: SQLite with WAL mode, stored in the Tauri app data directory as `gallery.db`. Thumbnails stored alongside as `thumbnails/`.
### Tauri events (backend → frontend)
| Event | Payload |
|-------|---------|
| `index-progress` | `IndexProgress` |
| `media-job-progress` | `MediaJobProgressEvent` |
| `indexed-images` | `IndexedImagesBatch` |
| `media-updated` | `ThumbnailBatch` |
| `caption-model-progress` | `CaptionModelProgress` |
| `tagger-model-progress` | `TaggerModelProgress` |
| `duplicate_scan_progress` | `[scanned, total]` |
### Key types
`ImageRecord` (mirrored in `store.ts` and `db.rs`) is the central data type. It carries embedding status, tagging status, caption data, and media metadata. The frontend type must stay in sync with the Rust struct serialization.
## Development notes
- Hot Reload is active during `dev:app` — do not restart the server for frontend changes. Restart only when adding Rust crates or changing Vite config.
- ML inference crates (`candle-*`, `ort`, `image`, `rayon`, `tokenizers`, `xxhash-rust`, `rusqlite`) use `opt-level = 3` in dev profile to keep inference performance acceptable.
- The caption worker is intentionally disabled (`lib.rs:73`) — the backend code is intact for future re-enabling.
- **Never use `any` type** in TypeScript — look up correct types.
+34 -92
View File
@@ -1,116 +1,58 @@
# Phokus
## Overview
A local-first desktop media library for browsing, filtering, and curating image and video folders.
Phokus is a Tauri desktop app for building a fast, local media library from folders on disk. It indexes images and videos, stores metadata in SQLite, and gives you a dense browsing workflow with filtering, favorites, ratings, and a lightbox preview.
## Features
The current app is optimized for:
- local folders instead of cloud import flows
- large visual libraries
- quick review and curation
- mixed image and video browsing
## Current features
- Add and remove media folders
- Background indexing with progress updates
- Browse all media or filter by folder
- Search by filename
- Filter by images, videos, or favorites
- Sort by modified date, name, or file size
- Grid density controls
- Lightbox preview with keyboard navigation
- Favorite and star-rating metadata saved in SQLite
- Virtualized/local-first architecture built on Tauri + React
- Add and remove media folders; background indexing with live progress
- Browse all media or filter by folder, type (image/video), favorites, or star rating
- **Filename search**, **semantic search** (`/s query`), and **tag search** (`/t tag`)
- **Similar image search** — find visually similar media by image or selected region
- **AI tagging** via WD tagger (ONNX, CPU/DirectML) with confidence threshold control
- **Explore view** — visual cluster map and tag cloud for browsing by theme
- **Duplicate finder** — scan for exact duplicates by file hash with bulk delete
- Lightbox preview with keyboard navigation, inline tag editing, and rating controls
- Sort by date, name, size, rating, or duration
- Grid density controls (compact / comfortable / detail)
## Supported formats
Images:
- `jpg`
- `jpeg`
- `png`
- `gif`
- `bmp`
- `tiff`
- `tif`
- `webp`
- `avif`
- `heic`
- `heif`
Videos:
- `mp4`
- `mov`
- `m4v`
- `webm`
| Images | Videos |
|--------|--------|
| jpg, jpeg, png, gif, bmp | mp4, mov, m4v |
| tiff, tif, webp, avif, heic, heif | webm |
## Stack
- Tauri 2
- React 19
- TypeScript
- Zustand
- Rust
- SQLite + `sqlite-vec`
- Vite
## Project structure
- `src/`: React UI, state, and components
- `src-tauri/src/commands.rs`: Tauri command surface
- `src-tauri/src/db.rs`: SQLite schema and queries
- `src-tauri/src/indexer.rs`: folder crawling and batch indexing
- `src-tauri/src/vector.rs`: vector table setup for future semantic workflows
- Tauri 2 + Rust backend
- React 19 + TypeScript + Zustand
- SQLite + `sqlite-vec` (vector search)
- ONNX Runtime (`ort`) for AI tagging
- Candle (Rust ML) for visual embeddings
- FFmpeg sidecar for video thumbnails and metadata
- Vite + Tailwind CSS v4
## Development
### Prerequisites
- Node.js 20+
- `pnpm`
- Rust toolchain
- Tauri system prerequisites for Windows
### Install
**Prerequisites:** Node.js 20+, pnpm, Rust toolchain, Tauri system prerequisites for Windows.
```bash
pnpm install
```
### Run in development
# Run with hot-reload (frontend + Rust)
pnpm dev:app
```bash
pnpm tauri dev
```
# Frontend only
pnpm dev:vite
### Build
```bash
pnpm tauri build
# Production build
pnpm build:app
```
## How it works
1. Add a folder from the sidebar or Library menu.
2. The Rust indexer walks the directory recursively.
3. Supported files are written into SQLite with metadata such as path, size, dimensions, media type, rating, and favorite state.
1. Add a folder from the sidebar — the Rust indexer walks it recursively.
2. Supported files are written to SQLite with metadata (path, dimensions, media type, etc.).
3. Background workers generate thumbnails, compute visual embeddings, and run AI tagging.
4. Progress events stream back to the UI while the gallery updates incrementally.
5. The gallery view loads media in pages and opens items in a lightbox for review.
## Notes
- This is currently a local desktop library, not a sync product.
- Search is filename-based right now.
- The vector table and embedding fields exist, but semantic search is not wired into the UI yet.
- Some visible UI copy may still use the old working name until the frontend text is updated.
## Positioning
The clearest product description today is:
> A local-first desktop media library for browsing, filtering, and curating image and video folders.
That description is more accurate than "gallery" alone and gives you a better base for future branding, onboarding copy, and a landing page.
5. Embeddings power semantic search and the similar images feature via an HNSW index.
+1 -3
View File
@@ -2,11 +2,9 @@
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Tauri + React + Typescript</title>
<title>Phokus</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
+6 -2
View File
@@ -4,8 +4,10 @@
"version": "0.1.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
"build:app": "tauri build",
"build:vite": "tsc && vite build",
"dev:app": "tauri dev",
"dev:vite": "vite",
"preview": "vite preview",
"tauri": "tauri"
},
@@ -15,6 +17,7 @@
"@tauri-apps/plugin-dialog": "^2.7.0",
"@tauri-apps/plugin-fs": "^2.5.0",
"@tauri-apps/plugin-opener": "^2",
"d3-force": "^3.0.0",
"framer-motion": "^12.38.0",
"react": "^19.1.0",
"react-dom": "^19.1.0",
@@ -23,6 +26,7 @@
"devDependencies": {
"@tailwindcss/vite": "^4.2.2",
"@tauri-apps/cli": "^2",
"@types/d3-force": "^3.0.10",
"@types/react": "^19.1.8",
"@types/react-dom": "^19.1.6",
"@vitejs/plugin-react": "^4.6.0",
+39
View File
@@ -23,6 +23,9 @@ importers:
'@tauri-apps/plugin-opener':
specifier: ^2
version: 2.5.3
d3-force:
specifier: ^3.0.0
version: 3.0.0
framer-motion:
specifier: ^12.38.0
version: 12.38.0(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
@@ -42,6 +45,9 @@ importers:
'@tauri-apps/cli':
specifier: ^2
version: 2.10.1
'@types/d3-force':
specifier: ^3.0.10
version: 3.0.10
'@types/react':
specifier: ^19.1.8
version: 19.2.14
@@ -662,6 +668,9 @@ packages:
'@types/babel__traverse@7.28.0':
resolution: {integrity: sha512-8PvcXf70gTDZBgt9ptxJ8elBeBjcLOAcOtoO/mPJjtji1+CdGbHgm77om1GrsPxsiE+uXIpNSK64UYaIwQXd4Q==}
'@types/d3-force@3.0.10':
resolution: {integrity: sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw==}
'@types/estree@1.0.8':
resolution: {integrity: sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==}
@@ -698,6 +707,22 @@ packages:
csstype@3.2.3:
resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==}
d3-dispatch@3.0.1:
resolution: {integrity: sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg==}
engines: {node: '>=12'}
d3-force@3.0.0:
resolution: {integrity: sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg==}
engines: {node: '>=12'}
d3-quadtree@3.0.1:
resolution: {integrity: sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw==}
engines: {node: '>=12'}
d3-timer@3.0.1:
resolution: {integrity: sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==}
engines: {node: '>=12'}
debug@4.4.3:
resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==}
engines: {node: '>=6.0'}
@@ -1448,6 +1473,8 @@ snapshots:
dependencies:
'@babel/types': 7.29.0
'@types/d3-force@3.0.10': {}
'@types/estree@1.0.8': {}
'@types/react-dom@19.2.3(@types/react@19.2.14)':
@@ -1486,6 +1513,18 @@ snapshots:
csstype@3.2.3: {}
d3-dispatch@3.0.1: {}
d3-force@3.0.0:
dependencies:
d3-dispatch: 3.0.1
d3-quadtree: 3.0.1
d3-timer: 3.0.1
d3-quadtree@3.0.1: {}
d3-timer@3.0.1: {}
debug@4.4.3:
dependencies:
ms: 2.1.3
+12
View File
@@ -0,0 +1,12 @@
[target.x86_64-pc-windows-msvc]
rustflags = [
# Disable MSVC incremental linking — avoids .ilk file corruption
# and removes one source of link failure on restart.
"-C", "link-arg=/INCREMENTAL:NO",
# Skip PDB generation entirely in dev builds.
# The .pdb file is the most common reason the linker fails after
# an unclean shutdown: the previous phokus.exe process holds the
# file open, so link.exe cannot write a new one → LNK error.
# Rust backtraces still work without MSVC PDBs.
"-C", "link-arg=/DEBUG:NONE",
]
+332
View File
@@ -61,6 +61,73 @@ dependencies = [
"libc",
]
[[package]]
name = "anndists"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8396b473aa0bceed68fb32462505387ea39fa47c7029417e0a49f10592b036"
dependencies = [
"anyhow",
"cfg-if",
"cpu-time",
"env_logger",
"lazy_static",
"log",
"num-traits",
"num_cpus",
"rayon",
]
[[package]]
name = "anstream"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
dependencies = [
"anstyle",
"anstyle-parse",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
"is_terminal_polyfill",
"utf8parse",
]
[[package]]
name = "anstyle"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anstyle-parse"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
dependencies = [
"utf8parse",
]
[[package]]
name = "anstyle-query"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "anstyle-wincon"
version = "3.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys 0.61.2",
]
[[package]]
name = "anyhow"
version = "1.0.102"
@@ -266,6 +333,15 @@ version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "bincode"
version = "1.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad"
dependencies = [
"serde",
]
[[package]]
name = "bit-set"
version = "0.8.0"
@@ -595,6 +671,12 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "cfg_aliases"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.10.0"
@@ -626,6 +708,12 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b"
[[package]]
name = "colorchoice"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "combine"
version = "4.6.7"
@@ -757,6 +845,16 @@ dependencies = [
"libc",
]
[[package]]
name = "cpu-time"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9e393a7668fe1fad3075085b86c781883000b4ede868f43627b34a87c8b7ded"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "cpufeatures"
version = "0.2.17"
@@ -874,6 +972,27 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "csv"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52cd9d68cf7efc6ddfaaee42e7288d3a99d613d4b50f76ce9827ae0c6e14f938"
dependencies = [
"csv-core",
"itoa",
"ryu",
"serde_core",
]
[[package]]
name = "csv-core"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "704a3c26996a80471189265814dbc2c257598b96b8a7feae2d31ace646bb9782"
dependencies = [
"memchr",
]
[[package]]
name = "ctor"
version = "0.2.9"
@@ -1331,12 +1450,35 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "env_filter"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32e90c2accc4b07a8456ea0debdc2e7587bdd890680d71173a15d4ae604f6eef"
dependencies = [
"log",
"regex",
]
[[package]]
name = "env_home"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7f84e12ccf0a7ddc17a6c41c93326024c42920d7ee630d04950e6926645c0fe"
[[package]]
name = "env_logger"
version = "0.11.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0621c04f2196ac3f488dd583365b9c09be011a4ab8b9f37248ffcc8f6198b56a"
dependencies = [
"anstream",
"anstyle",
"env_filter",
"jiff",
"log",
]
[[package]]
name = "equivalent"
version = "1.0.2"
@@ -2319,6 +2461,8 @@ version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash 0.1.5",
]
@@ -2395,6 +2539,31 @@ version = "1.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec9d92d097f4749b64e8cc33d924d9f40a2d4eb91402b458014b781f5733d60f"
[[package]]
name = "hnsw_rs"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43a5258f079b97bf2e8311ff9579e903c899dcbac0d9a138d62e9a066778bd07"
dependencies = [
"anndists",
"anyhow",
"bincode",
"cfg-if",
"cpu-time",
"env_logger",
"hashbrown 0.15.5",
"indexmap 2.13.1",
"lazy_static",
"log",
"mmap-rs",
"num-traits",
"num_cpus",
"parking_lot",
"rand 0.9.2",
"rayon",
"serde",
]
[[package]]
name = "html5ever"
version = "0.29.1"
@@ -2798,6 +2967,12 @@ dependencies = [
"once_cell",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
[[package]]
name = "itertools"
version = "0.14.0"
@@ -2836,6 +3011,30 @@ dependencies = [
"system-deps",
]
[[package]]
name = "jiff"
version = "0.2.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a3546dc96b6d42c5f24902af9e2538e82e39ad350b0c766eb3fbf2d8f3d8359"
dependencies = [
"jiff-static",
"log",
"portable-atomic",
"portable-atomic-util",
"serde_core",
]
[[package]]
name = "jiff-static"
version = "0.2.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a8c8b344124222efd714b73bb41f8b5120b27a7cc1c75593a6ff768d9d05aa4"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "jni"
version = "0.21.1"
@@ -2937,6 +3136,12 @@ dependencies = [
"selectors 0.24.0",
]
[[package]]
name = "lazy_static"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]]
name = "leb128fmt"
version = "0.1.0"
@@ -3088,6 +3293,15 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
[[package]]
name = "mach2"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d640282b302c0bb0a2a8e0233ead9035e3bed871f0b7e81fe4a1ec829765db44"
dependencies = [
"libc",
]
[[package]]
name = "macro_rules_attribute"
version = "0.2.2"
@@ -3214,6 +3428,23 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "mmap-rs"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ecce9d566cb9234ae3db9e249c8b55665feaaf32b0859ff1e27e310d2beb3d8"
dependencies = [
"bitflags 2.11.0",
"combine",
"libc",
"mach2",
"nix",
"sysctl",
"thiserror 2.0.18",
"widestring",
"windows 0.48.0",
]
[[package]]
name = "monostate"
version = "0.1.18"
@@ -3335,6 +3566,18 @@ version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
[[package]]
name = "nix"
version = "0.30.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6"
dependencies = [
"bitflags 2.11.0",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]]
name = "nodrop"
version = "0.1.14"
@@ -3613,6 +3856,12 @@ version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "once_cell_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]]
name = "onig"
version = "6.5.1"
@@ -4009,11 +4258,14 @@ dependencies = [
"candle-nn",
"candle-transformers",
"chrono",
"csv",
"fast_image_resize",
"ffmpeg-sidecar",
"hf-hub",
"hnsw_rs",
"image",
"log",
"memmap2",
"ort",
"r2d2",
"r2d2_sqlite",
@@ -4030,9 +4282,11 @@ dependencies = [
"tauri-plugin-opener",
"tokenizers",
"tokio",
"ureq",
"uuid",
"walkdir",
"xxhash-rust",
"zip 4.6.1",
]
[[package]]
@@ -6551,6 +6805,12 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]]
name = "utf8parse"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.23.0"
@@ -6914,6 +7174,12 @@ dependencies = [
"winsafe",
]
[[package]]
name = "widestring"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72069c3113ab32ab29e5584db3c6ec55d416895e60715417b5b883a357c3e471"
[[package]]
name = "winapi"
version = "0.3.9"
@@ -6960,6 +7226,15 @@ dependencies = [
"windows-version",
]
[[package]]
name = "windows"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e686886bc078bc1b0b600cac0147aadb815089b6e4da64016cbd754b6342700f"
dependencies = [
"windows-targets 0.48.5",
]
[[package]]
name = "windows"
version = "0.61.3"
@@ -7212,6 +7487,21 @@ dependencies = [
"windows_x86_64_msvc 0.42.2",
]
[[package]]
name = "windows-targets"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c"
dependencies = [
"windows_aarch64_gnullvm 0.48.5",
"windows_aarch64_msvc 0.48.5",
"windows_i686_gnu 0.48.5",
"windows_i686_msvc 0.48.5",
"windows_x86_64_gnu 0.48.5",
"windows_x86_64_gnullvm 0.48.5",
"windows_x86_64_msvc 0.48.5",
]
[[package]]
name = "windows-targets"
version = "0.52.6"
@@ -7278,6 +7568,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8"
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8"
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.52.6"
@@ -7296,6 +7592,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43"
[[package]]
name = "windows_aarch64_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc"
[[package]]
name = "windows_aarch64_msvc"
version = "0.52.6"
@@ -7314,6 +7616,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f"
[[package]]
name = "windows_i686_gnu"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e"
[[package]]
name = "windows_i686_gnu"
version = "0.52.6"
@@ -7344,6 +7652,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060"
[[package]]
name = "windows_i686_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406"
[[package]]
name = "windows_i686_msvc"
version = "0.52.6"
@@ -7362,6 +7676,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36"
[[package]]
name = "windows_x86_64_gnu"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e"
[[package]]
name = "windows_x86_64_gnu"
version = "0.52.6"
@@ -7380,6 +7700,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.52.6"
@@ -7398,6 +7724,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0"
[[package]]
name = "windows_x86_64_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538"
[[package]]
name = "windows_x86_64_msvc"
version = "0.52.6"
+54 -1
View File
@@ -27,6 +27,7 @@ rusqlite = { version = "0.32", features = ["bundled"] }
r2d2 = "0.8"
r2d2_sqlite = "0.25"
sqlite-vec = "=0.1.9"
hnsw_rs = "0.3.4"
image = { version = "0.25", default-features = false, features = ["jpeg", "png", "webp", "gif", "bmp", "tiff"] }
fast_image_resize = { version = "6.0.0", features = ["image"] }
walkdir = "2"
@@ -38,10 +39,62 @@ anyhow = "1"
log = "0.4"
ffmpeg-sidecar = "2.5.0"
xxhash-rust = { version = "0.8", features = ["xxh3"] }
memmap2 = "0.9"
sysinfo = "0.38.4"
candle-core = { version = "0.10.2", features = ["cuda"] }
candle-nn = { version = "0.10.2", features = ["cuda"] }
candle-transformers = { version = "0.10.2", features = ["cuda"] }
hf-hub = { version = "0.5.0", default-features = false, features = ["ureq", "native-tls"] }
tokenizers = "0.22.1"
ort = { version = "2.0.0-rc.12", default-features = false, features = ["std", "ndarray", "download-binaries", "copy-dylibs", "load-dynamic", "api-24", "tls-native"] }
ort = { version = "2.0.0-rc.12", default-features = false, features = ["std", "ndarray", "download-binaries", "copy-dylibs", "load-dynamic", "directml", "api-24", "tls-native"] }
ureq = { version = "3.3.0", default-features = false, features = ["native-tls"] }
zip = { version = "4.6.1", default-features = false, features = ["deflate"] }
csv = "1"
# ── Dev-mode performance ────────────────────────────────────────────────────
# opt-level=1 on the main crate keeps incremental compile short.
# Only the packages that are genuine hot-path bottlenecks in dev get opt-level=3
# (using "*" caused cargo to recheck all dependency fingerprints too aggressively,
# which caused spurious full rebuilds and linker conflicts).
[profile.dev]
opt-level = 1
# ML inference — without opt these run 20-50× slower than release
[profile.dev.package.candle-core]
opt-level = 3
[profile.dev.package.candle-nn]
opt-level = 3
[profile.dev.package.candle-transformers]
opt-level = 3
# ONNX runtime (WD tagger)
[profile.dev.package.ort]
opt-level = 3
[profile.dev.package.ort-sys]
opt-level = 3
# Image decode/resize workers
[profile.dev.package.image]
opt-level = 3
[profile.dev.package.fast_image_resize]
opt-level = 3
# Parallel work scheduler
[profile.dev.package.rayon]
opt-level = 3
[profile.dev.package.rayon-core]
opt-level = 3
# Tokenisation (embedding model pre-processing)
[profile.dev.package.tokenizers]
opt-level = 3
# Hashing (duplicate finder)
[profile.dev.package.xxhash-rust]
opt-level = 3
# SQLite (frequent db calls in workers)
[profile.dev.package.rusqlite]
opt-level = 3
[profile.dev.package.libsqlite3-sys]
opt-level = 3
+475 -85
View File
@@ -1,18 +1,54 @@
use anyhow::Result;
use hf_hub::{api::sync::Api, Repo, RepoType};
use image::{imageops::FilterType, ImageReader};
use ort::ep;
use ort::session::SessionInputValue;
use ort::session::SessionOutputs;
use ort::session::{builder::GraphOptimizationLevel, Session};
use ort::value::{Shape, Tensor};
use serde::Serialize;
use serde::{Deserialize, Serialize};
use std::borrow::Cow;
use std::io::{Cursor, Read};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::OnceLock;
use std::time::Instant;
use tokenizers::Tokenizer;
pub const FLORENCE_MODEL_ID: &str = "onnx-community/Florence-2-base-ft";
pub const FLORENCE_CAPTION_MODEL_NAME: &str = "florence-2-base-ft-onnx-q4";
const ONNX_RUNTIME_NUGET_URL: &str =
"https://www.nuget.org/api/v2/package/Microsoft.ML.OnnxRuntime.DirectML/1.24.2";
const DIRECTML_NUGET_URL: &str =
"https://www.nuget.org/api/v2/package/Microsoft.AI.DirectML/1.15.4";
const ONNX_RUNTIME_DLL_FILE: &str = "onnxruntime/onnxruntime.dll";
const ONNX_RUNTIME_PROVIDERS_DLL_FILE: &str = "onnxruntime/onnxruntime_providers_shared.dll";
const DIRECTML_DLL_FILE: &str = "onnxruntime/DirectML.dll";
const CAPTION_ACCELERATION_FILE: &str = "settings/caption_acceleration.txt";
const CAPTION_DETAIL_FILE: &str = "settings/caption_detail.txt";
const ONNX_RUNTIME_FILES: &[(&str, &str, &str)] = &[
(
ONNX_RUNTIME_DLL_FILE,
ONNX_RUNTIME_NUGET_URL,
"runtimes/win-x64/native/onnxruntime.dll",
),
(
ONNX_RUNTIME_PROVIDERS_DLL_FILE,
ONNX_RUNTIME_NUGET_URL,
"runtimes/win-x64/native/onnxruntime_providers_shared.dll",
),
(
DIRECTML_DLL_FILE,
DIRECTML_NUGET_URL,
"bin/x64-win/DirectML.dll",
),
];
const REQUIRED_FILES: &[&str] = &[
ONNX_RUNTIME_DLL_FILE,
ONNX_RUNTIME_PROVIDERS_DLL_FILE,
DIRECTML_DLL_FILE,
"config.json",
"generation_config.json",
"preprocessor_config.json",
@@ -21,10 +57,81 @@ const REQUIRED_FILES: &[&str] = &[
"special_tokens_map.json",
"onnx/vision_encoder_fp16.onnx",
"onnx/encoder_model_q4.onnx",
"onnx/decoder_model_q4.onnx",
"onnx/decoder_model_merged_q4.onnx",
"onnx/embed_tokens_fp16.onnx",
];
static ORT_RUNTIME_INIT: OnceLock<std::result::Result<(), String>> = OnceLock::new();
/// Set to `true` by `set_caption_acceleration` so the caption worker loop
/// knows to drop its cached `FlorenceCaptioner` and reload with the new EP.
pub static CAPTION_SESSION_DIRTY: AtomicBool = AtomicBool::new(false);
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum CaptionAcceleration {
Auto,
Cpu,
Directml,
}
impl CaptionAcceleration {
fn as_str(self) -> &'static str {
match self {
Self::Auto => "auto",
Self::Cpu => "cpu",
Self::Directml => "directml",
}
}
}
impl Default for CaptionAcceleration {
fn default() -> Self {
Self::Auto
}
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum CaptionDetail {
Short,
Detailed,
Paragraph,
}
impl CaptionDetail {
fn as_str(self) -> &'static str {
match self {
Self::Short => "short",
Self::Detailed => "detailed",
Self::Paragraph => "paragraph",
}
}
fn prompt(self) -> &'static str {
match self {
Self::Short => "What does the image describe?",
Self::Detailed => "Describe in detail what is shown in the image.",
Self::Paragraph => "Describe with a paragraph what is shown in the image.",
}
}
fn max_new_tokens(self) -> usize {
match self {
Self::Short => 32,
Self::Detailed => 56,
Self::Paragraph => 96,
}
}
}
impl Default for CaptionDetail {
fn default() -> Self {
Self::Paragraph
}
}
#[derive(Serialize)]
pub struct CaptionModelStatus {
pub model_id: &'static str,
@@ -45,6 +152,8 @@ pub struct CaptionModelProgress {
#[derive(Serialize)]
pub struct CaptionRuntimeProbe {
pub ready: bool,
pub acceleration: CaptionAcceleration,
pub detail: CaptionDetail,
pub tokenizer_vocab_size: usize,
pub sessions: Vec<CaptionRuntimeSessionProbe>,
}
@@ -61,6 +170,7 @@ pub struct CaptionVisionProbe {
pub input_shape: Vec<i64>,
pub output_shape: Vec<i64>,
pub output_values: usize,
pub acceleration: CaptionAcceleration,
}
#[derive(Clone)]
@@ -71,9 +181,11 @@ struct TensorData {
pub struct FlorenceCaptioner {
tokenizer: Tokenizer,
caption_detail: CaptionDetail,
vision_session: Session,
embed_session: Session,
encoder_session: Session,
decoder_prefill_session: Session,
decoder_session: Session,
}
@@ -81,6 +193,54 @@ pub fn model_dir(app_data_dir: &Path) -> PathBuf {
app_data_dir.join("models").join("florence-2-base-ft")
}
pub fn caption_acceleration(app_data_dir: &Path) -> CaptionAcceleration {
let path = app_data_dir.join(CAPTION_ACCELERATION_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return CaptionAcceleration::default();
};
match value.trim().to_ascii_lowercase().as_str() {
"cpu" => CaptionAcceleration::Cpu,
"directml" => CaptionAcceleration::Directml,
_ => CaptionAcceleration::Auto,
}
}
pub fn set_caption_acceleration(
app_data_dir: &Path,
acceleration: CaptionAcceleration,
) -> Result<CaptionAcceleration> {
let path = app_data_dir.join(CAPTION_ACCELERATION_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, acceleration.as_str())?;
CAPTION_SESSION_DIRTY.store(true, Ordering::Relaxed);
Ok(acceleration)
}
pub fn caption_detail(app_data_dir: &Path) -> CaptionDetail {
let path = app_data_dir.join(CAPTION_DETAIL_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return CaptionDetail::default();
};
match value.trim().to_ascii_lowercase().as_str() {
"short" => CaptionDetail::Short,
"paragraph" => CaptionDetail::Paragraph,
_ => CaptionDetail::Detailed,
}
}
pub fn set_caption_detail(app_data_dir: &Path, detail: CaptionDetail) -> Result<CaptionDetail> {
let path = app_data_dir.join(CAPTION_DETAIL_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, detail.as_str())?;
Ok(detail)
}
pub fn caption_model_status(app_data_dir: &Path) -> CaptionModelStatus {
let local_dir = model_dir(app_data_dir);
let missing_files = REQUIRED_FILES
@@ -133,9 +293,20 @@ pub fn prepare_caption_model_with_progress(
if let Some(parent) = destination.parent() {
std::fs::create_dir_all(parent)?;
}
if ONNX_RUNTIME_FILES
.iter()
.any(|(runtime_file, _, _)| runtime_file == file)
{
download_onnx_runtime_files(&local_dir)?;
completed_files = REQUIRED_FILES
.iter()
.filter(|file| local_dir.join(file).exists())
.count();
} else {
let cached = repo.get(file)?;
std::fs::copy(cached, destination)?;
completed_files += 1;
}
emit_progress(CaptionModelProgress {
total_files: REQUIRED_FILES.len(),
completed_files,
@@ -172,21 +343,40 @@ pub fn probe_caption_runtime(app_data_dir: &Path) -> Result<CaptionRuntimeProbe>
}
let local_dir = model_dir(app_data_dir);
ensure_onnx_runtime(&local_dir)?;
let tokenizer =
Tokenizer::from_file(local_dir.join("tokenizer.json")).map_err(anyhow::Error::msg)?;
let sessions = [
"onnx/vision_encoder_fp16.onnx",
let acceleration = caption_acceleration(app_data_dir);
// For the vision encoder (the only session that runs on the GPU EP) we
// create an actual ORT session so we can verify which EP was actually
// loaded, rather than just echoing the settings file.
let vision_session = probe_vision_session(
&local_dir.join("onnx/vision_encoder_fp16.onnx"),
acceleration,
)?;
// The remaining entries are DLLs and CPU-only text/decoder models — probe
// them with the lightweight file-size check as before.
let other_files = [
"onnxruntime/onnxruntime.dll",
"onnxruntime/onnxruntime_providers_shared.dll",
"onnxruntime/DirectML.dll",
"onnx/embed_tokens_fp16.onnx",
"onnx/encoder_model_q4.onnx",
"onnx/decoder_model_q4.onnx",
"onnx/decoder_model_merged_q4.onnx",
]
.into_iter()
.map(|file| probe_session(file, &local_dir.join(file)))
.collect::<Result<Vec<_>>>()?;
];
let mut sessions = vec![vision_session];
for file in other_files {
sessions.push(probe_session(file, &local_dir.join(file))?);
}
Ok(CaptionRuntimeProbe {
ready: true,
acceleration,
detail: caption_detail(app_data_dir),
tokenizer_vocab_size: tokenizer.get_vocab_size(false),
sessions,
})
@@ -202,11 +392,17 @@ pub fn probe_caption_vision(app_data_dir: &Path, image_path: &Path) -> Result<Ca
}
let local_dir = model_dir(app_data_dir);
ensure_onnx_runtime(&local_dir)?;
let pixels = preprocess_image(image_path)?;
let input_shape = vec![1, 3, 768, 768];
let input = Tensor::from_array(([1usize, 3, 768, 768], pixels.into_boxed_slice()))
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut session = create_session(&local_dir.join("onnx/vision_encoder_fp16.onnx"))?;
let acceleration = caption_acceleration(app_data_dir);
let mut session = create_session(
&local_dir.join("onnx/vision_encoder_fp16.onnx"),
acceleration,
true,
)?;
let outputs = session
.run(ort::inputs! {
"pixel_values" => input
@@ -220,6 +416,7 @@ pub fn probe_caption_vision(app_data_dir: &Path, image_path: &Path) -> Result<Ca
input_shape,
output_shape: output_shape.to_vec(),
output_values: output_values.len(),
acceleration,
})
}
@@ -230,6 +427,7 @@ pub fn generate_caption(app_data_dir: &Path, image_path: &Path) -> Result<String
impl FlorenceCaptioner {
pub fn new(app_data_dir: &Path) -> Result<Self> {
let started_at = Instant::now();
let status = caption_model_status(app_data_dir);
if !status.ready {
anyhow::bail!(
@@ -239,48 +437,92 @@ impl FlorenceCaptioner {
}
let local_dir = model_dir(app_data_dir);
ensure_onnx_runtime(&local_dir)?;
let tokenizer =
Tokenizer::from_file(local_dir.join("tokenizer.json")).map_err(anyhow::Error::msg)?;
let caption_detail = caption_detail(app_data_dir);
let vision_session = create_session(&local_dir.join("onnx/vision_encoder_fp16.onnx"))?;
let embed_session = create_session(&local_dir.join("onnx/embed_tokens_fp16.onnx"))?;
let encoder_session = create_session(&local_dir.join("onnx/encoder_model_q4.onnx"))?;
let decoder_session = create_session(&local_dir.join("onnx/decoder_model_merged_q4.onnx"))?;
let sessions_started_at = Instant::now();
let acceleration = caption_acceleration(app_data_dir);
let vision_session = create_session(
&local_dir.join("onnx/vision_encoder_fp16.onnx"),
acceleration,
true,
)?;
let embed_session = create_session(
&local_dir.join("onnx/embed_tokens_fp16.onnx"),
acceleration,
false,
)?;
let encoder_session = create_session(
&local_dir.join("onnx/encoder_model_q4.onnx"),
acceleration,
false,
)?;
let decoder_prefill_session = create_session(
&local_dir.join("onnx/decoder_model_q4.onnx"),
acceleration,
false,
)?;
let decoder_session = create_session(
&local_dir.join("onnx/decoder_model_merged_q4.onnx"),
acceleration,
false,
)?;
println!(
"Florence sessions loaded in {:?} with {:?} acceleration (total init {:?})",
sessions_started_at.elapsed(),
acceleration,
started_at.elapsed()
);
Ok(Self {
tokenizer,
caption_detail,
vision_session,
embed_session,
encoder_session,
decoder_prefill_session,
decoder_session,
})
}
pub fn generate(&mut self, image_path: &Path) -> Result<String> {
let started_at = Instant::now();
println!("Florence caption started: {}", image_path.display());
let image_features = run_vision_encoder(&mut self.vision_session, image_path)?;
println!("Florence vision encoder done in {:?}", started_at.elapsed());
let prompt_ids = self
.tokenizer
.encode("What does the image describe?", false)
.encode(self.caption_detail.prompt(), false)
.map_err(anyhow::Error::msg)?
.get_ids()
.iter()
.map(|id| i64::from(*id))
.collect::<Vec<_>>();
let prompt_embeds = run_token_embedder(&mut self.embed_session, &prompt_ids)?;
let encoder_embeds = concatenate_sequence_embeddings(&prompt_embeds, &image_features)?;
println!(
"Florence token embeddings done in {:?}",
started_at.elapsed()
);
let encoder_embeds = concatenate_sequence_embeddings(&image_features, &prompt_embeds)?;
let encoder_attention_mask = vec![1_i64; encoder_embeds.shape[1] as usize];
let encoder_hidden_states = run_encoder(
&mut self.encoder_session,
&encoder_embeds,
&encoder_attention_mask,
)?;
println!("Florence encoder done in {:?}", started_at.elapsed());
let generated_ids = run_decoder(
&mut self.decoder_prefill_session,
&mut self.decoder_session,
&mut self.embed_session,
&encoder_hidden_states,
&encoder_attention_mask,
self.caption_detail.max_new_tokens(),
)?;
println!("Florence decoder done in {:?}", started_at.elapsed());
let generated_u32 = generated_ids
.into_iter()
@@ -330,6 +572,26 @@ fn probe_session(file: &'static str, path: &Path) -> Result<CaptionRuntimeSessio
],
vec!["logits".to_string(), "present_key_values".to_string()],
),
"onnxruntime/onnxruntime.dll" => (
vec!["OrtGetApiBase".to_string()],
vec!["ORT DirectML 1.24.2".to_string()],
),
"onnxruntime/onnxruntime_providers_shared.dll" => (
vec!["providers".to_string()],
vec!["shared provider bridge".to_string()],
),
"onnxruntime/DirectML.dll" => (
vec!["DirectX 12".to_string()],
vec!["DirectML 1.15.4".to_string()],
),
"onnx/decoder_model_q4.onnx" => (
vec![
"encoder_attention_mask".to_string(),
"encoder_hidden_states".to_string(),
"inputs_embeds".to_string(),
],
vec!["logits".to_string(), "present_key_values".to_string()],
),
_ => (Vec::new(), Vec::new()),
};
@@ -340,6 +602,111 @@ fn probe_session(file: &'static str, path: &Path) -> Result<CaptionRuntimeSessio
})
}
/// Create an actual ORT session for the vision encoder and return a probe
/// entry that reflects which execution provider was successfully loaded.
/// This is the only session that can legitimately run on DirectML; all
/// text/decoder sessions intentionally use CPU only.
fn probe_vision_session(
path: &Path,
acceleration: CaptionAcceleration,
) -> Result<CaptionRuntimeSessionProbe> {
let metadata = std::fs::metadata(path)?;
if metadata.len() == 0 {
anyhow::bail!("{} is empty", path.display());
}
// Try to create the session with the requested acceleration. If DirectML
// was requested but fails we report what actually loaded.
let loaded_acceleration = match acceleration {
CaptionAcceleration::Cpu => {
create_session(path, CaptionAcceleration::Cpu, false)?;
CaptionAcceleration::Cpu
}
CaptionAcceleration::Auto => {
// `fail_silently` — session will fall back to CPU if DirectML
// is unavailable; we detect this by trying Directml explicitly.
let directml_ok = create_session(path, CaptionAcceleration::Directml, true).is_ok();
if directml_ok {
CaptionAcceleration::Directml
} else {
create_session(path, CaptionAcceleration::Cpu, false)?;
CaptionAcceleration::Cpu
}
}
CaptionAcceleration::Directml => {
create_session(path, CaptionAcceleration::Directml, true)?;
CaptionAcceleration::Directml
}
};
Ok(CaptionRuntimeSessionProbe {
file: "onnx/vision_encoder_fp16.onnx",
inputs: vec!["pixel_values".to_string()],
outputs: vec![format!("image_features [EP: {:?}]", loaded_acceleration)],
})
}
pub fn ensure_onnx_runtime(local_dir: &Path) -> Result<()> {
let dll_path = local_dir.join(ONNX_RUNTIME_DLL_FILE);
ORT_RUNTIME_INIT
.get_or_init(|| {
if !dll_path.exists() {
return Err(format!(
"ONNX Runtime DLL is missing: {}",
dll_path.display()
));
}
ort::environment::init_from(&dll_path)
.map_err(|error| error.to_string())?
.with_name("phokus-florence")
.commit();
Ok(())
})
.clone()
.map_err(anyhow::Error::msg)
}
fn download_onnx_runtime_files(local_dir: &Path) -> Result<()> {
if !cfg!(target_os = "windows") {
anyhow::bail!(
"Florence-2 ONNX Runtime download is currently configured for Windows builds"
);
}
for (destination_file, source_url, archive_path) in ONNX_RUNTIME_FILES {
let destination = local_dir.join(destination_file);
download_nuget_file(source_url, archive_path, &destination)?;
}
Ok(())
}
fn download_nuget_file(source_url: &str, archive_path: &str, destination: &Path) -> Result<()> {
let mut response = ureq::get(source_url)
.call()
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut bytes = Vec::new();
response
.body_mut()
.as_reader()
.read_to_end(&mut bytes)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut archive = zip::ZipArchive::new(Cursor::new(bytes))?;
let mut dll = archive.by_name(archive_path)?;
if let Some(parent) = destination.parent() {
std::fs::create_dir_all(parent)?;
}
let temp_destination = destination.with_extension("tmp");
{
let mut file = std::fs::File::create(&temp_destination)?;
std::io::copy(&mut dll, &mut file)?;
}
std::fs::rename(temp_destination, destination)?;
Ok(())
}
fn run_vision_encoder(session: &mut Session, image_path: &Path) -> Result<TensorData> {
let pixels = preprocess_image(image_path)?;
let input = Tensor::from_array(([1usize, 3, 768, 768], pixels.into_boxed_slice()))
@@ -387,24 +754,50 @@ fn run_encoder(
}
fn run_decoder(
decoder_prefill_session: &mut Session,
decoder_session: &mut Session,
embed_session: &mut Session,
encoder_hidden_states: &TensorData,
encoder_attention_mask: &[i64],
max_new_tokens: usize,
) -> Result<Vec<i64>> {
const DECODER_LAYERS: usize = 6;
const DECODER_HEADS: usize = 12;
const HEAD_DIM: usize = 64;
const DECODER_START_TOKEN_ID: i64 = 2;
const FORCED_BOS_TOKEN_ID: i64 = 0;
const EOS_TOKEN_ID: i64 = 2;
const MAX_NEW_TOKENS: usize = 32;
let mut generated = Vec::new();
let mut next_input_id = DECODER_START_TOKEN_ID;
let mut past: Vec<TensorData> = Vec::new();
let mut use_cache_branch = false;
let encoder_attention_mask_tensor = Tensor::from_array((
[1usize, encoder_attention_mask.len()],
encoder_attention_mask.to_vec().into_boxed_slice(),
))
.map_err(|error| anyhow::anyhow!("{error}"))?;
let encoder_hidden_states_tensor = tensor_from_data(encoder_hidden_states)?;
let prefill_inputs_embeds = run_token_embedder(embed_session, &[DECODER_START_TOKEN_ID])?;
let prefill_inputs_embeds_tensor = tensor_from_data(&prefill_inputs_embeds)?;
let prefill_started_at = Instant::now();
let prefill_outputs = decoder_prefill_session
.run(ort::inputs! {
"encoder_attention_mask" => encoder_attention_mask_tensor,
"encoder_hidden_states" => encoder_hidden_states_tensor,
"inputs_embeds" => prefill_inputs_embeds_tensor
})
.map_err(|error| anyhow::anyhow!("{error}"))?;
println!(
"Florence decoder prefill done in {:?}",
prefill_started_at.elapsed()
);
let _prefill_logits = tensor_data(&prefill_outputs[0])?;
let mut next_input_id = FORCED_BOS_TOKEN_ID;
let encoder_kv = collect_present_key_values(&prefill_outputs, DECODER_LAYERS)?;
let mut decoder_kv = encoder_kv.clone();
for _ in 0..max_new_tokens {
if next_input_id == EOS_TOKEN_ID {
break;
}
generated.push(next_input_id);
for _ in 0..MAX_NEW_TOKENS {
let inputs_embeds = run_token_embedder(embed_session, &[next_input_id])?;
let encoder_attention_mask_tensor = Tensor::from_array((
[1usize, encoder_attention_mask.len()],
@@ -413,8 +806,7 @@ fn run_decoder(
.map_err(|error| anyhow::anyhow!("{error}"))?;
let encoder_hidden_states_tensor = tensor_from_data(encoder_hidden_states)?;
let inputs_embeds_tensor = tensor_from_data(&inputs_embeds)?;
let use_cache_branch_tensor =
Tensor::from_array(([1usize], vec![use_cache_branch].into_boxed_slice()))
let use_cache_branch_tensor = Tensor::from_array(([1usize], vec![true].into_boxed_slice()))
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut inputs = ort::inputs! {
@@ -424,92 +816,82 @@ fn run_decoder(
"use_cache_branch" => use_cache_branch_tensor
};
if past.is_empty() {
for layer in 0..DECODER_LAYERS {
push_tensor_input(
&mut inputs,
format!("past_key_values.{layer}.decoder.key"),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]),
decoder_kv[layer * 4].clone(),
)?;
push_tensor_input(
&mut inputs,
format!("past_key_values.{layer}.decoder.value"),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]),
decoder_kv[layer * 4 + 1].clone(),
)?;
push_tensor_input(
&mut inputs,
format!("past_key_values.{layer}.encoder.key"),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]),
encoder_kv[layer * 4 + 2].clone(),
)?;
push_tensor_input(
&mut inputs,
format!("past_key_values.{layer}.encoder.value"),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]),
encoder_kv[layer * 4 + 3].clone(),
)?;
}
} else {
for layer in 0..DECODER_LAYERS {
for cache_name in [
"decoder.key",
"decoder.value",
"encoder.key",
"encoder.value",
] {
let past_index = layer * 4
+ match cache_name {
"decoder.key" => 0,
"decoder.value" => 1,
"encoder.key" => 2,
"encoder.value" => 3,
_ => unreachable!(),
};
push_tensor_input(
&mut inputs,
format!("past_key_values.{layer}.{cache_name}"),
past[past_index].clone(),
)?;
}
}
}
let outputs = decoder_session
.run(inputs)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let logits = tensor_data(&outputs["logits"])?;
let token_id = argmax_last_token(&logits)?;
if token_id == EOS_TOKEN_ID {
break;
}
generated.push(token_id);
next_input_id = token_id;
past.clear();
for layer in 0..DECODER_LAYERS {
for cache_name in [
"decoder.key",
"decoder.value",
"encoder.key",
"encoder.value",
] {
past.push(tensor_data(
&outputs[format!("present.{layer}.{cache_name}").as_str()],
)?);
}
}
use_cache_branch = true;
next_input_id = argmax_last_token(&logits)?;
decoder_kv = collect_present_key_values(&outputs, DECODER_LAYERS)?;
}
println!("Florence decoder produced {} token(s)", generated.len());
Ok(generated)
}
fn create_session(path: &Path) -> Result<Session> {
fn create_session(
path: &Path,
acceleration: CaptionAcceleration,
allow_directml: bool,
) -> Result<Session> {
let builder = Session::builder().map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_optimization_level(GraphOptimizationLevel::Level3)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut builder = builder
let use_directml = allow_directml
&& matches!(
acceleration,
CaptionAcceleration::Auto | CaptionAcceleration::Directml
);
let builder = builder
.with_memory_pattern(!use_directml)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_parallel_execution(false)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_intra_threads(1)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut builder = match acceleration {
CaptionAcceleration::Cpu => builder,
CaptionAcceleration::Auto if use_directml => builder
.with_execution_providers([ep::DirectML::default().build().fail_silently()])
.unwrap_or_else(|error| error.recover()),
CaptionAcceleration::Directml if use_directml => builder
.with_execution_providers([ep::DirectML::default().build().error_on_failure()])
.map_err(|error| anyhow::anyhow!("{error}"))?,
CaptionAcceleration::Auto | CaptionAcceleration::Directml => {
// `allow_directml` is false for the 4 text/decoder sessions —
// they intentionally run on CPU regardless of the setting.
println!(
"Florence: using CPU for {} (DirectML disabled for this session type)",
path.display()
);
builder
}
};
let session = builder
.commit_from_file(path)
.map_err(|error| anyhow::anyhow!("{error}"))?;
@@ -537,6 +919,24 @@ fn concatenate_sequence_embeddings(text: &TensorData, image: &TensorData) -> Res
})
}
fn collect_present_key_values(
outputs: &SessionOutputs<'_>,
layers: usize,
) -> Result<Vec<TensorData>> {
let expected = layers * 4;
if outputs.len() < expected + 1 {
anyhow::bail!(
"Decoder returned {} output(s), expected at least {}",
outputs.len(),
expected + 1
);
}
(1..=expected)
.map(|index| tensor_data(&outputs[index]))
.collect::<Result<Vec<_>>>()
}
fn tensor_data(value: &ort::value::DynValue) -> Result<TensorData> {
let (shape, values) = value
.try_extract_tensor::<f32>()
@@ -608,13 +1008,3 @@ fn preprocess_image(image_path: &Path) -> Result<Vec<f32>> {
Ok(pixel_values)
}
impl TensorData {
fn zeros(shape: Vec<i64>) -> Self {
let values_len = shape.iter().map(|value| (*value).max(0) as usize).product();
Self {
shape,
values: vec![0.0; values_len],
}
}
}
+750 -11
View File
@@ -1,7 +1,12 @@
use crate::captioner::{self, CaptionModelStatus, CaptionRuntimeProbe, CaptionVisionProbe};
use crate::db::{self, DbPool, Folder, FolderJobProgress, ImageRecord};
use crate::captioner::{
self, CaptionAcceleration, CaptionDetail, CaptionModelStatus, CaptionRuntimeProbe,
CaptionVisionProbe,
};
use crate::db::{self, DbPool, ExploreTagEntry, Folder, FolderJobProgress, ImageRecord, ImageTag};
use crate::embedder;
use crate::hnsw_index;
use crate::indexer;
use crate::tagger::{self, TaggerAcceleration, TaggerModelStatus, TaggerRuntimeProbe};
use crate::vector;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
@@ -17,12 +22,21 @@ pub struct ImagesPage {
pub limit: i64,
}
#[derive(Serialize)]
pub struct SimilarImagesPage {
pub images: Vec<ImageRecord>,
pub offset: usize,
pub limit: usize,
pub has_more: bool,
}
#[derive(Deserialize)]
pub struct GetImagesParams {
pub folder_id: Option<i64>,
pub search: Option<String>,
pub media_kind: Option<String>,
pub favorites_only: Option<bool>,
pub rating_min: Option<i64>,
pub embedding_failed_only: Option<bool>,
pub sort: Option<String>,
pub offset: Option<i64>,
@@ -39,12 +53,29 @@ pub struct UpdateImageDetailsParams {
#[derive(Deserialize)]
pub struct FindSimilarImagesParams {
pub image_id: i64,
pub folder_id: Option<i64>,
pub offset: Option<usize>,
pub limit: Option<usize>,
pub threshold: Option<f32>,
}
#[derive(Deserialize)]
pub struct FindSimilarByRegionParams {
pub image_id: i64,
/// Normalized crop rect (0.01.0).
pub crop_x: f32,
pub crop_y: f32,
pub crop_w: f32,
pub crop_h: f32,
pub folder_id: Option<i64>,
pub offset: Option<usize>,
pub limit: Option<usize>,
}
#[derive(Deserialize)]
pub struct DebugSimilarImagesParams {
pub image_id: i64,
pub folder_id: Option<i64>,
pub limit: Option<usize>,
}
@@ -72,6 +103,26 @@ pub struct QueueCaptionJobsParams {
pub image_id: Option<i64>,
}
#[derive(Deserialize)]
pub struct ClearCaptionJobsParams {
pub folder_id: Option<i64>,
}
#[derive(Deserialize)]
pub struct ResetCaptionsParams {
pub folder_id: Option<i64>,
}
#[derive(Deserialize)]
pub struct SetCaptionAccelerationParams {
pub acceleration: CaptionAcceleration,
}
#[derive(Deserialize)]
pub struct SetCaptionDetailParams {
pub detail: CaptionDetail,
}
#[derive(Deserialize)]
pub struct ProbeCaptionImageParams {
pub image_id: i64,
@@ -88,9 +139,56 @@ pub struct SemanticSearchParams {
pub folder_id: Option<i64>,
pub media_kind: Option<String>,
pub favorites_only: Option<bool>,
pub rating_min: Option<i64>,
pub limit: Option<usize>,
}
#[derive(Deserialize)]
pub struct TagSearchParams {
pub query: String,
pub folder_id: Option<i64>,
pub media_kind: Option<String>,
pub favorites_only: Option<bool>,
pub rating_min: Option<i64>,
pub limit: Option<usize>,
}
#[derive(Deserialize)]
pub struct GetExploreTagsParams {
pub folder_id: Option<i64>,
pub limit: Option<usize>,
}
#[derive(Deserialize)]
pub struct SearchTagsAutocompleteParams {
pub query: String,
pub folder_id: Option<i64>,
pub limit: Option<usize>,
}
#[derive(Serialize, Deserialize)]
pub struct DuplicateGroup {
pub file_hash: String,
pub file_size: u64,
pub images: Vec<ImageRecord>,
}
#[derive(Serialize)]
pub struct DuplicateScanCache {
pub groups: Vec<DuplicateGroup>,
pub scanned_at: i64,
}
#[derive(Deserialize)]
pub struct DeleteImagesFromDiskParams {
pub image_ids: Vec<i64>,
}
#[derive(Deserialize)]
pub struct GetImagesByIdsParams {
pub image_ids: Vec<i64>,
}
#[tauri::command]
pub async fn add_folder(
app: AppHandle,
@@ -161,6 +259,7 @@ pub async fn get_images(
let search = params.search.as_deref();
let media_kind = params.media_kind.as_deref();
let favorites_only = params.favorites_only.unwrap_or(false);
let rating_min = params.rating_min.unwrap_or(0);
let embedding_failed_only = params.embedding_failed_only.unwrap_or(false);
let total = db::count_images(
@@ -169,6 +268,7 @@ pub async fn get_images(
search,
media_kind,
favorites_only,
rating_min,
embedding_failed_only,
)
.map_err(|e| e.to_string())?;
@@ -179,6 +279,7 @@ pub async fn get_images(
search,
media_kind,
favorites_only,
rating_min,
embedding_failed_only,
sort,
offset,
@@ -228,16 +329,102 @@ pub async fn reindex_folder(
pub async fn find_similar_images(
db: State<'_, DbState>,
params: FindSimilarImagesParams,
) -> Result<Vec<ImageRecord>, String> {
) -> Result<SimilarImagesPage, String> {
let conn = db.get().map_err(|e| e.to_string())?;
let limit = params.limit.unwrap_or(32);
let offset = params.offset.unwrap_or(0);
let threshold = params.threshold.unwrap_or(0.24);
if !vector::has_image_vector(&conn, params.image_id).map_err(|e| e.to_string())? {
db::repair_embedding_consistency(&conn).map_err(|e| e.to_string())?;
return Ok(Vec::new());
return Ok(SimilarImagesPage {
images: Vec::new(),
offset,
limit,
has_more: false,
});
}
let image_ids =
vector::find_similar_image_ids(&conn, params.image_id, limit).map_err(|e| e.to_string())?;
db::get_images_by_ids(&conn, &image_ids).map_err(|e| e.to_string())
let matches = hnsw_index::find_similar_image_matches(
&conn,
params.image_id,
params.folder_id,
threshold,
offset,
limit + 1,
)
.map_err(|e| e.to_string())?;
let has_more = matches.len() > limit;
let image_ids = matches
.into_iter()
.take(limit)
.map(|(image_id, _)| image_id)
.collect::<Vec<_>>();
let images = db::get_images_by_ids(&conn, &image_ids).map_err(|e| e.to_string())?;
Ok(SimilarImagesPage {
images,
offset,
limit,
has_more,
})
}
#[tauri::command]
pub async fn find_similar_by_region(
db: State<'_, DbState>,
params: FindSimilarByRegionParams,
) -> Result<SimilarImagesPage, String> {
let conn = db.get().map_err(|e| e.to_string())?;
let limit = params.limit.unwrap_or(32);
let offset = params.offset.unwrap_or(0);
// Look up the source image path
let image = db::get_image_by_id(&conn, params.image_id).map_err(|e| e.to_string())?;
let image_path = std::path::Path::new(&image.path);
// Embed the cropped region in-memory (no temp file needed)
let embedder = embedder::ClipImageEmbedder::new().map_err(|e| e.to_string())?;
let embedding = embedder
.embed_image_crop(
image_path,
params.crop_x,
params.crop_y,
params.crop_w,
params.crop_h,
)
.map_err(|e| e.to_string())?;
// Search for similar images using the crop embedding
let image_ids = match params.folder_id {
Some(folder_id) => vector::search_image_ids_by_embedding_in_folder(
&conn,
&embedding,
folder_id,
Some(params.image_id),
offset + limit + 1,
)
.map_err(|e| e.to_string())?,
None => {
let mut ids = vector::search_image_ids_by_embedding(&conn, &embedding, offset + limit + 1)
.map_err(|e| e.to_string())?;
// Exclude the source image from global results
ids.retain(|&id| id != params.image_id);
ids
}
};
let has_more = image_ids.len() > offset + limit;
let page_ids = image_ids
.into_iter()
.skip(offset)
.take(limit)
.collect::<Vec<_>>();
let images = db::get_images_by_ids(&conn, &page_ids).map_err(|e| e.to_string())?;
Ok(SimilarImagesPage {
images,
offset,
limit,
has_more,
})
}
#[derive(Serialize)]
@@ -258,7 +445,8 @@ pub async fn debug_similar_images(
let vector_count = vector::count_image_vectors(&conn).map_err(|e| e.to_string())?;
let has_vector = vector::has_image_vector(&conn, params.image_id).map_err(|e| e.to_string())?;
let similar_ids = if has_vector {
vector::find_similar_image_ids(&conn, params.image_id, limit).map_err(|e| e.to_string())?
vector::find_similar_image_ids(&conn, params.image_id, limit, params.folder_id)
.map_err(|e| e.to_string())?
} else {
Vec::new()
};
@@ -301,10 +489,31 @@ pub async fn semantic_search_images(
if params.favorites_only.unwrap_or(false) {
images.retain(|image| image.favorite);
}
if let Some(rating_min) = params.rating_min {
images.retain(|image| image.rating >= rating_min);
}
Ok(images)
}
#[tauri::command]
pub async fn search_images_by_tag(
db: State<'_, DbState>,
params: TagSearchParams,
) -> Result<Vec<ImageRecord>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::search_images_by_tag(
&conn,
&params.query,
params.folder_id,
params.media_kind.as_deref(),
params.favorites_only.unwrap_or(false),
params.rating_min.unwrap_or(0),
params.limit.unwrap_or(64),
)
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn set_generated_caption(
db: State<'_, DbState>,
@@ -332,6 +541,36 @@ pub async fn get_caption_model_status(app: AppHandle) -> Result<CaptionModelStat
Ok(captioner::caption_model_status(&app_dir))
}
#[tauri::command]
pub async fn get_caption_acceleration(app: AppHandle) -> Result<CaptionAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(captioner::caption_acceleration(&app_dir))
}
#[tauri::command]
pub async fn set_caption_acceleration(
app: AppHandle,
params: SetCaptionAccelerationParams,
) -> Result<CaptionAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
captioner::set_caption_acceleration(&app_dir, params.acceleration).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_caption_detail(app: AppHandle) -> Result<CaptionDetail, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(captioner::caption_detail(&app_dir))
}
#[tauri::command]
pub async fn set_caption_detail(
app: AppHandle,
params: SetCaptionDetailParams,
) -> Result<CaptionDetail, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
captioner::set_caption_detail(&app_dir, params.detail).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn prepare_caption_model(app: AppHandle) -> Result<CaptionModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
@@ -441,11 +680,31 @@ pub async fn queue_caption_jobs(
}
}
#[tauri::command]
pub async fn clear_caption_jobs(
db: State<'_, DbState>,
params: ClearCaptionJobsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::clear_caption_jobs(&conn, params.folder_id).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn reset_generated_captions(
db: State<'_, DbState>,
params: ResetCaptionsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::reset_generated_captions(&conn, params.folder_id).map_err(|e| e.to_string())
}
#[derive(Serialize, Deserialize)]
pub struct TagCloudEntry {
pub count: usize,
pub representative_image_id: i64,
pub thumbnail_path: Option<String>,
#[serde(default)]
pub image_ids: Vec<i64>,
}
fn fnv_hash_ids(ids: &[i64]) -> u64 {
@@ -494,10 +753,14 @@ pub async fn get_tag_cloud(
.map_err(|e| e.to_string())?
{
if let Ok(entries) = serde_json::from_str::<Vec<TagCloudEntry>>(&json) {
// Reject cache entries written before image_ids were tracked — they all
// have empty image_ids which causes "No media found" when a cluster is opened.
if entries.iter().all(|e| !e.image_ids.is_empty()) {
return Ok(entries);
}
}
}
}
// Cache miss — run k-means
let ids: Vec<i64> = embeddings_with_ids.iter().map(|(id, _)| *id).collect();
@@ -522,6 +785,13 @@ pub async fn get_tag_cloud(
}
let centroid = &centroids[ci];
let cluster_ids = points
.iter()
.enumerate()
.filter(|(i, _)| assignments[*i] == ci)
.map(|(i, _)| ids[i])
.collect::<Vec<_>>();
let best_id = points
.iter()
.enumerate()
@@ -539,6 +809,7 @@ pub async fn get_tag_cloud(
count,
representative_image_id: best_id,
thumbnail_path,
image_ids: cluster_ids,
});
}
@@ -550,6 +821,224 @@ pub async fn get_tag_cloud(
Ok(entries)
}
#[tauri::command]
pub async fn get_explore_tags(
db: State<'_, DbState>,
params: GetExploreTagsParams,
) -> Result<Vec<ExploreTagEntry>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_explore_tags(&conn, params.folder_id, params.limit.unwrap_or(48)).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn search_tags_autocomplete(
db: State<'_, DbState>,
params: SearchTagsAutocompleteParams,
) -> Result<Vec<ExploreTagEntry>, String> {
if params.query.trim().is_empty() {
return Ok(vec![]);
}
let conn = db.get().map_err(|e| e.to_string())?;
db::search_tags_autocomplete(&conn, &params.query, params.folder_id, params.limit.unwrap_or(10))
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn find_duplicates(
app: AppHandle,
db: State<'_, DbState>,
folder_id: Option<i64>,
) -> Result<Vec<DuplicateGroup>, String> {
let records = {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_all_image_paths(&conn, folder_id).map_err(|e| e.to_string())?
};
let total = records.len();
let _ = app.emit("duplicate_scan_progress", (0usize, total));
// Two-phase detection. No full-file read at any point.
//
// Phase 1 — stat:
// Read only file metadata (size). Files with a unique size cannot be
// duplicates; discard them immediately. Zero file content read.
//
// Phase 2 — sample hash:
// For each size-matched candidate, read four evenly-spaced 16 KB windows
// (head, 33%, 66%, tail) and hash them together. Total I/O per file is
// capped at 64 KB regardless of how large the file is.
//
// For small files (≤ 64 KB) the windows cover the whole file, so the
// hash is exact. For large files, matching all four windows at different
// offsets is effectively impossible for natural photo/video content unless
// the files are genuinely identical — eliminating the need for a full read.
let app_hash = app.clone();
let pairs: Vec<(u64, u64, i64)> = tokio::task::spawn_blocking(move || {
use memmap2::Mmap;
use rayon::prelude::*;
use std::collections::HashMap;
use std::sync::atomic::{AtomicUsize, Ordering};
use xxhash_rust::xxh3::{xxh3_64, Xxh3};
const WINDOW: usize = 16 * 1024; // 16 KB per sample window
const N: usize = 4; // windows at 0%, 33%, 66%, 100%
const COVERED: usize = WINDOW * N; // 64 KB total; also full-coverage threshold
// Hash four evenly-spaced 16 KB windows. For files ≤ 64 KB this is
// equivalent to hashing the entire file.
fn sample(mmap: &[u8]) -> u64 {
if mmap.len() <= COVERED {
return xxh3_64(mmap);
}
let mut h = Xxh3::new();
for i in 0..N {
let pos = (mmap.len() - WINDOW) * i / (N - 1);
h.update(&mmap[pos..pos + WINDOW]);
}
h.digest()
}
// ── Phase 1: stat ─────────────────────────────────────────────────
let sized: Vec<(i64, String, u64)> = records
.par_iter()
.filter_map(|r| {
let size = std::fs::metadata(&r.path).ok()?.len();
if size == 0 { return None; }
Some((r.id, r.path.clone(), size))
})
.collect();
let mut by_size: HashMap<u64, Vec<usize>> = HashMap::new();
for (i, (_, _, size)) in sized.iter().enumerate() {
by_size.entry(*size).or_default().push(i);
}
let candidates: Vec<usize> = by_size
.into_values()
.filter(|g| g.len() > 1)
.flatten()
.collect();
if candidates.is_empty() {
return vec![];
}
// ── Phase 2: sample hash ──────────────────────────────────────────
let c_total = candidates.len();
let _ = app_hash.emit("duplicate_scan_progress", (0usize, c_total));
let counter = AtomicUsize::new(0);
candidates
.par_iter()
.filter_map(|&idx| {
let (id, path, size) = &sized[idx];
let file = std::fs::File::open(path).ok()?;
// SAFETY: read-only; no external truncation expected during scan.
let mmap = unsafe { Mmap::map(&file).ok()? };
let hash = sample(&mmap);
let done = counter.fetch_add(1, Ordering::Relaxed) + 1;
if done % 100 == 0 || done == c_total {
let _ = app_hash.emit("duplicate_scan_progress", (done, c_total));
}
Some((hash, *size, *id))
})
.collect()
})
.await
.map_err(|e| e.to_string())?;
let mut size_hash_map: std::collections::HashMap<(u64, u64), Vec<i64>> = std::collections::HashMap::new();
for (hash, file_size, id) in pairs {
size_hash_map.entry((hash, file_size)).or_default().push(id);
}
// Resolve image records for each duplicate group
let conn = db.get().map_err(|e| e.to_string())?;
let mut groups: Vec<DuplicateGroup> = size_hash_map
.into_iter()
.filter(|(_, ids)| ids.len() > 1)
.filter_map(|((hash, file_size), ids)| {
let images = db::get_images_by_ids(&conn, &ids).ok()?;
Some(DuplicateGroup {
file_hash: format!("{:016x}", hash),
file_size,
images,
})
})
.collect();
// Largest duplicates first — wastes the most space
groups.sort_by(|a, b| b.file_size.cmp(&a.file_size));
// Persist results so they survive restart — best-effort, ignore errors.
let folder_scope = match folder_id {
Some(id) => format!("folder:{}", id),
None => "all".to_string(),
};
if let Ok(json) = serde_json::to_string(&groups) {
let _ = db::set_duplicate_scan_cache(&conn, &folder_scope, &json);
}
Ok(groups)
}
#[tauri::command]
pub async fn load_duplicate_scan_cache(
db: State<'_, DbState>,
folder_id: Option<i64>,
) -> Result<Option<DuplicateScanCache>, String> {
let folder_scope = match folder_id {
Some(id) => format!("folder:{}", id),
None => "all".to_string(),
};
let conn = db.get().map_err(|e| e.to_string())?;
match db::get_duplicate_scan_cache(&conn, &folder_scope).map_err(|e| e.to_string())? {
Some((json, scanned_at)) => {
let groups: Vec<DuplicateGroup> =
serde_json::from_str(&json).map_err(|e| e.to_string())?;
Ok(Some(DuplicateScanCache { groups, scanned_at }))
}
None => Ok(None),
}
}
#[tauri::command]
pub async fn delete_images_from_disk(
db: State<'_, DbState>,
params: DeleteImagesFromDiskParams,
) -> Result<usize, String> {
if params.image_ids.is_empty() {
return Ok(0);
}
let conn = db.get().map_err(|e| e.to_string())?;
// Collect paths before deleting DB rows
let records = db::get_all_image_paths(&conn, None).map_err(|e| e.to_string())?;
let id_set: std::collections::HashSet<i64> = params.image_ids.iter().copied().collect();
let paths: Vec<String> = records
.into_iter()
.filter(|r| id_set.contains(&r.id))
.map(|r| r.path)
.collect();
db::delete_images_by_ids(&conn, &params.image_ids).map_err(|e| e.to_string())?;
let mut deleted = 0usize;
for path in &paths {
if std::fs::remove_file(path).is_ok() {
deleted += 1;
}
}
Ok(deleted)
}
#[tauri::command]
pub async fn get_images_by_ids(
db: State<'_, DbState>,
params: GetImagesByIdsParams,
) -> Result<Vec<ImageRecord>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_images_by_ids(&conn, &params.image_ids).map_err(|e| e.to_string())
}
// ── k-means with cosine similarity (all vectors assumed to be unit-normalized) ──
fn dot(a: &[f32], b: &[f32]) -> f32 {
@@ -670,11 +1159,27 @@ pub struct FolderWorkerStates {
pub metadata_paused: bool,
pub embedding_paused: bool,
pub caption_paused: bool,
pub tagging_paused: bool,
}
#[tauri::command]
pub async fn set_worker_paused(worker: String, folder_id: i64, paused: bool) -> Result<(), String> {
pub async fn set_worker_paused(
db: State<'_, DbState>,
worker: String,
folder_id: i64,
paused: bool,
) -> Result<(), String> {
indexer::set_worker_paused(&worker, folder_id, paused);
if worker == "caption" && paused {
let conn = db.get().map_err(|e| e.to_string())?;
db::requeue_processing_caption_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
if worker == "tagging" && paused {
let conn = db.get().map_err(|e| e.to_string())?;
db::requeue_processing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
Ok(())
}
@@ -684,8 +1189,7 @@ pub async fn get_worker_states(folder_ids: Vec<i64>) -> Result<Vec<FolderWorkerS
Ok(folder_ids
.into_iter()
.map(|folder_id| {
let state =
states
let state = states
.get(&folder_id)
.copied()
.unwrap_or(indexer::FolderWorkerPausedState {
@@ -693,6 +1197,7 @@ pub async fn get_worker_states(folder_ids: Vec<i64>) -> Result<Vec<FolderWorkerS
metadata: false,
embedding: false,
caption: false,
tagging: false,
});
FolderWorkerStates {
folder_id,
@@ -700,7 +1205,241 @@ pub async fn get_worker_states(folder_ids: Vec<i64>) -> Result<Vec<FolderWorkerS
metadata_paused: state.metadata,
embedding_paused: state.embedding,
caption_paused: state.caption,
tagging_paused: state.tagging,
}
})
.collect())
}
// ---------------------------------------------------------------------------
// Tagger commands
// ---------------------------------------------------------------------------
#[derive(Deserialize)]
pub struct SetTaggerAccelerationParams {
pub acceleration: TaggerAcceleration,
}
#[derive(Deserialize)]
pub struct SetTaggerThresholdParams {
pub threshold: f32,
}
#[derive(Deserialize)]
pub struct SetTaggerBatchSizeParams {
pub batch_size: usize,
}
#[derive(Deserialize)]
pub struct QueueTaggingJobsParams {
pub folder_id: Option<i64>,
pub folder_ids: Option<Vec<i64>>,
pub image_id: Option<i64>,
}
#[derive(Deserialize)]
pub struct ClearTaggingJobsParams {
pub folder_id: Option<i64>,
pub folder_ids: Option<Vec<i64>>,
}
#[derive(Deserialize)]
pub struct GetImageTagsParams {
pub image_id: i64,
}
#[derive(Deserialize)]
pub struct AddUserTagParams {
pub image_id: i64,
pub tag: String,
}
#[derive(Deserialize)]
pub struct RemoveTagParams {
pub tag_id: i64,
}
#[tauri::command]
pub async fn get_tagger_model_status(app: AppHandle) -> Result<TaggerModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_model_status(&app_dir))
}
#[tauri::command]
pub async fn get_tagger_acceleration(app: AppHandle) -> Result<TaggerAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_acceleration(&app_dir))
}
#[tauri::command]
pub async fn set_tagger_acceleration(
app: AppHandle,
params: SetTaggerAccelerationParams,
) -> Result<TaggerAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tagger::set_tagger_acceleration(&app_dir, params.acceleration).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn probe_tagger_runtime(app: AppHandle) -> Result<TaggerRuntimeProbe, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tauri::async_runtime::spawn_blocking(move || tagger::probe_tagger_runtime(&app_dir))
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_tagger_threshold(app: AppHandle) -> Result<f32, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_threshold(&app_dir))
}
#[tauri::command]
pub async fn set_tagger_threshold(
app: AppHandle,
params: SetTaggerThresholdParams,
) -> Result<f32, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tagger::set_tagger_threshold(&app_dir, params.threshold).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_tagger_batch_size(app: AppHandle) -> Result<usize, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_batch_size(&app_dir))
}
#[tauri::command]
pub async fn set_tagger_batch_size(
app: AppHandle,
params: SetTaggerBatchSizeParams,
) -> Result<usize, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tagger::set_tagger_batch_size(&app_dir, params.batch_size).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn prepare_tagger_model(app: AppHandle) -> Result<TaggerModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tauri::async_runtime::spawn_blocking(move || {
let app = app.clone();
tagger::prepare_tagger_model_with_progress(&app_dir, move |progress| {
let _ = app.emit("tagger-model-progress", progress);
})
})
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn delete_tagger_model(app: AppHandle) -> Result<TaggerModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tauri::async_runtime::spawn_blocking(move || tagger::delete_tagger_model(&app_dir))
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn queue_tagging_jobs(
app: AppHandle,
db: State<'_, DbState>,
params: QueueTaggingJobsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
let requested_folder_ids = params.folder_ids.unwrap_or_default();
let (total, folder_ids) = match (params.folder_id, params.image_id, requested_folder_ids.is_empty()) {
(_, Some(image_id), _) => {
db::enqueue_tagging_job(&conn, image_id).map_err(|e| e.to_string())?;
// Look up just this image's folder_id rather than fetching all folders
let image = db::get_image_by_id(&conn, image_id).map_err(|e| e.to_string())?;
(1usize, vec![image.folder_id])
}
(Some(folder_id), None, _) => {
let n = db::enqueue_missing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
(n, vec![folder_id])
}
(None, None, false) => {
let mut total = 0usize;
for &folder_id in &requested_folder_ids {
total += db::enqueue_missing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
(total, requested_folder_ids)
}
(None, None, true) => {
let folders = db::get_folders(&conn).map_err(|e| e.to_string())?;
let folder_ids: Vec<i64> = folders.iter().map(|f| f.id).collect();
let mut total = 0usize;
for &folder_id in &folder_ids {
total += db::enqueue_missing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
(total, folder_ids)
}
};
drop(conn);
indexer::emit_folder_job_progress(&app, db.inner(), &folder_ids, true);
Ok(total)
}
#[tauri::command]
pub async fn clear_tagging_jobs(
app: AppHandle,
db: State<'_, DbState>,
params: ClearTaggingJobsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
let requested_folder_ids = params.folder_ids.unwrap_or_default();
let (n, folder_ids): (usize, Vec<i64>) = match (params.folder_id, requested_folder_ids.is_empty()) {
(Some(id), _) => (
db::clear_tagging_jobs(&conn, Some(id)).map_err(|e| e.to_string())?,
vec![id],
),
(None, false) => {
let mut total = 0usize;
for &folder_id in &requested_folder_ids {
total += db::clear_tagging_jobs(&conn, Some(folder_id)).map_err(|e| e.to_string())?;
}
(total, requested_folder_ids)
}
(None, true) => (
db::clear_tagging_jobs(&conn, None).map_err(|e| e.to_string())?,
db::get_folders(&conn)
.map_err(|e| e.to_string())?
.into_iter()
.map(|f| f.id)
.collect(),
),
};
drop(conn);
indexer::emit_folder_job_progress(&app, db.inner(), &folder_ids, true);
Ok(n)
}
#[tauri::command]
pub async fn get_image_tags(
db: State<'_, DbState>,
params: GetImageTagsParams,
) -> Result<Vec<ImageTag>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_image_tags(&conn, params.image_id).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn add_user_tag(
db: State<'_, DbState>,
params: AddUserTagParams,
) -> Result<ImageTag, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::add_user_tag(&conn, params.image_id, &params.tag).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn remove_tag(db: State<'_, DbState>, params: RemoveTagParams) -> Result<(), String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::remove_tag(&conn, params.tag_id).map_err(|e| e.to_string())
}
+701 -9
View File
@@ -61,6 +61,10 @@ pub struct ImageRecord {
pub caption_model: Option<String>,
pub caption_updated_at: Option<String>,
pub caption_error: Option<String>,
pub ai_rating: Option<String>,
pub ai_tagger_model: Option<String>,
pub ai_tagged_at: Option<String>,
pub ai_tagger_error: Option<String>,
}
#[allow(dead_code)]
@@ -100,6 +104,32 @@ pub struct CaptionJob {
pub path: String,
}
#[derive(Debug, Clone)]
pub struct TaggingJob {
pub image_id: i64,
pub folder_id: i64,
pub path: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ImageTag {
pub id: i64,
pub image_id: i64,
pub tag: String,
pub source: String,
pub ai_model: Option<String>,
pub confidence: Option<f64>,
pub created_at: String,
}
#[derive(Debug, Clone, Serialize)]
pub struct ExploreTagEntry {
pub tag: String,
pub count: i64,
pub representative_image_id: i64,
pub thumbnail_path: Option<String>,
}
#[derive(Debug, Clone)]
pub struct IndexedMediaEntry {
pub id: i64,
@@ -120,6 +150,9 @@ pub struct FolderJobProgress {
pub caption_pending: i64,
pub caption_ready: i64,
pub caption_failed: i64,
pub tagging_pending: i64,
pub tagging_ready: i64,
pub tagging_failed: i64,
}
pub fn create_pool(db_path: &Path) -> Result<DbPool> {
@@ -195,6 +228,26 @@ pub fn migrate(conn: &Connection) -> Result<()> {
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS tagging_jobs (
image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE,
status TEXT NOT NULL DEFAULT 'pending',
attempts INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS image_tags (
id INTEGER PRIMARY KEY AUTOINCREMENT,
image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
tag TEXT NOT NULL,
source TEXT NOT NULL DEFAULT 'user',
ai_model TEXT,
confidence REAL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(image_id, tag)
);
CREATE TABLE IF NOT EXISTS tag_cloud_cache (
folder_scope TEXT PRIMARY KEY,
image_ids_hash INTEGER NOT NULL,
@@ -202,12 +255,22 @@ pub fn migrate(conn: &Connection) -> Result<()> {
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS duplicate_scan_cache (
folder_scope TEXT PRIMARY KEY,
scanned_at INTEGER NOT NULL,
groups_json TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_images_folder_id ON images(folder_id);
CREATE INDEX IF NOT EXISTS idx_images_modified_at ON images(modified_at);
CREATE INDEX IF NOT EXISTS idx_embedding_jobs_status ON embedding_jobs(status);
CREATE INDEX IF NOT EXISTS idx_thumbnail_jobs_status ON thumbnail_jobs(status);
CREATE INDEX IF NOT EXISTS idx_metadata_jobs_status ON metadata_jobs(status);
CREATE INDEX IF NOT EXISTS idx_caption_jobs_status ON caption_jobs(status);
CREATE INDEX IF NOT EXISTS idx_tagging_jobs_status ON tagging_jobs(status);
CREATE INDEX IF NOT EXISTS idx_image_tags_image_id ON image_tags(image_id);
CREATE INDEX IF NOT EXISTS idx_image_tags_source ON image_tags(source);
CREATE INDEX IF NOT EXISTS idx_image_tags_tag ON image_tags(tag);
",
)?;
@@ -237,6 +300,10 @@ pub fn migrate(conn: &Connection) -> Result<()> {
ensure_column(conn, "images", "caption_model", "TEXT")?;
ensure_column(conn, "images", "caption_updated_at", "TEXT")?;
ensure_column(conn, "images", "caption_error", "TEXT")?;
ensure_column(conn, "images", "ai_rating", "TEXT")?;
ensure_column(conn, "images", "ai_tagger_model", "TEXT")?;
ensure_column(conn, "images", "ai_tagged_at", "TEXT")?;
ensure_column(conn, "images", "ai_tagger_error", "TEXT")?;
vector::migrate(conn)?;
Ok(())
@@ -257,8 +324,8 @@ pub fn insert_folder(conn: &Connection, path: &str, name: &str) -> Result<i64> {
pub fn upsert_image(conn: &Connection, img: &ImageRecord) -> Result<i64> {
let id = conn.query_row(
"INSERT INTO images (folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type, media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error, favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error, generated_caption, caption_model, caption_updated_at, caption_error)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19, ?20, ?21, ?22, ?23, ?24, ?25, ?26)
"INSERT INTO images (folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type, media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error, favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error, generated_caption, caption_model, caption_updated_at, caption_error, ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19, ?20, ?21, ?22, ?23, ?24, ?25, ?26, ?27, ?28, ?29, ?30)
ON CONFLICT(path) DO UPDATE SET
folder_id = excluded.folder_id,
filename = excluded.filename,
@@ -311,6 +378,10 @@ pub fn upsert_image(conn: &Connection, img: &ImageRecord) -> Result<i64> {
img.caption_model,
img.caption_updated_at,
img.caption_error,
img.ai_rating,
img.ai_tagger_model,
img.ai_tagged_at,
img.ai_tagger_error,
],
|row| row.get(0),
)?;
@@ -433,6 +504,13 @@ pub fn reset_inflight_jobs(conn: &Connection) -> Result<()> {
"UPDATE caption_jobs SET status = 'pending' WHERE status = 'processing'",
[],
)?;
conn.execute(
"UPDATE tagging_jobs SET status = 'pending' WHERE status = 'processing'",
[],
)?;
// Delete any rows that were cancelled before the previous shutdown so
// they don't silently linger in the DB across restarts.
conn.execute("DELETE FROM tagging_jobs WHERE status = 'cancelled'", [])?;
Ok(())
}
@@ -493,6 +571,110 @@ pub fn requeue_caption_jobs(conn: &Connection, image_ids: &[i64]) -> Result<()>
Ok(())
}
pub fn requeue_processing_caption_jobs_for_folder(
conn: &Connection,
folder_id: i64,
) -> Result<usize> {
conn.execute(
"UPDATE caption_jobs
SET status = 'pending', updated_at = datetime('now')
WHERE status = 'processing'
AND image_id IN (
SELECT id FROM images WHERE folder_id = ?1
)",
[folder_id],
)
.map_err(Into::into)
}
pub fn clear_caption_jobs(conn: &Connection, folder_id: Option<i64>) -> Result<usize> {
match folder_id {
Some(folder_id) => {
let deleted = conn.execute(
"DELETE FROM caption_jobs
WHERE image_id IN (
SELECT id FROM images WHERE folder_id = ?1
)",
[folder_id],
)?;
conn.execute(
"UPDATE images
SET caption_error = NULL
WHERE folder_id = ?1
AND generated_caption IS NULL",
[folder_id],
)?;
Ok(deleted)
}
None => {
let deleted = conn.execute("DELETE FROM caption_jobs", [])?;
conn.execute(
"UPDATE images
SET caption_error = NULL
WHERE generated_caption IS NULL",
[],
)?;
Ok(deleted)
}
}
}
pub fn reset_generated_captions(conn: &Connection, folder_id: Option<i64>) -> Result<usize> {
let image_ids = match folder_id {
Some(folder_id) => {
let mut stmt = conn.prepare(
"SELECT id FROM images WHERE folder_id = ?1 AND generated_caption IS NOT NULL",
)?;
let rows = stmt.query_map([folder_id], |row| row.get::<_, i64>(0))?;
rows.collect::<rusqlite::Result<Vec<_>>>()?
}
None => {
let mut stmt =
conn.prepare("SELECT id FROM images WHERE generated_caption IS NOT NULL")?;
let rows = stmt.query_map([], |row| row.get::<_, i64>(0))?;
rows.collect::<rusqlite::Result<Vec<_>>>()?
}
};
let tx = conn.unchecked_transaction()?;
for image_id in &image_ids {
vector::delete_caption_embedding(&tx, *image_id)?;
}
match folder_id {
Some(folder_id) => {
tx.execute(
"UPDATE images
SET generated_caption = NULL,
caption_model = NULL,
caption_updated_at = NULL,
caption_error = NULL
WHERE folder_id = ?1",
[folder_id],
)?;
tx.execute(
"DELETE FROM caption_jobs
WHERE image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[folder_id],
)?;
}
None => {
tx.execute(
"UPDATE images
SET generated_caption = NULL,
caption_model = NULL,
caption_updated_at = NULL,
caption_error = NULL",
[],
)?;
tx.execute("DELETE FROM caption_jobs", [])?;
}
}
tx.commit()?;
Ok(image_ids.len())
}
pub fn enqueue_missing_caption_jobs_for_folder(conn: &Connection, folder_id: i64) -> Result<usize> {
let inserted = conn.execute(
"INSERT INTO caption_jobs (image_id, status, attempts, last_error, created_at, updated_at)
@@ -791,9 +973,36 @@ pub fn get_folder_job_progress(conn: &Connection, folder_id: i64) -> Result<Fold
)?;
let caption_failed = conn.query_row(
"SELECT COUNT(*)
FROM caption_jobs j
JOIN images i ON i.id = j.image_id
WHERE i.folder_id = ?1 AND j.status = 'failed'",
[folder_id],
|row| row.get(0),
)?;
let tagging_pending = conn.query_row(
"SELECT COUNT(*)
FROM tagging_jobs j
JOIN images i ON i.id = j.image_id
WHERE i.folder_id = ?1 AND j.status IN ('pending', 'processing')",
[folder_id],
|row| row.get(0),
)?;
let tagging_ready = conn.query_row(
"SELECT COUNT(*)
FROM images
WHERE folder_id = ?1 AND caption_error IS NOT NULL",
WHERE folder_id = ?1 AND ai_tagged_at IS NOT NULL",
[folder_id],
|row| row.get(0),
)?;
let tagging_failed = conn.query_row(
"SELECT COUNT(*)
FROM tagging_jobs j
JOIN images i ON i.id = j.image_id
WHERE i.folder_id = ?1 AND j.status = 'failed'",
[folder_id],
|row| row.get(0),
)?;
@@ -808,6 +1017,9 @@ pub fn get_folder_job_progress(conn: &Connection, folder_id: i64) -> Result<Fold
caption_pending,
caption_ready,
caption_failed,
tagging_pending,
tagging_ready,
tagging_failed,
})
}
@@ -1046,7 +1258,8 @@ pub fn update_image_details(
"SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type,
media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error,
favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error,
generated_caption, caption_model, caption_updated_at, caption_error
generated_caption, caption_model, caption_updated_at, caption_error,
ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error
FROM images
WHERE id = ?1",
[image_id],
@@ -1060,7 +1273,8 @@ pub fn get_image_by_id(conn: &Connection, image_id: i64) -> Result<ImageRecord>
"SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type,
media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error,
favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error,
generated_caption, caption_model, caption_updated_at, caption_error
generated_caption, caption_model, caption_updated_at, caption_error,
ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error
FROM images
WHERE id = ?1",
[image_id],
@@ -1102,6 +1316,7 @@ pub fn get_images(
search: Option<&str>,
media_kind: Option<&str>,
favorites_only: bool,
rating_min: i64,
embedding_failed_only: bool,
sort: &str,
offset: i64,
@@ -1114,6 +1329,8 @@ pub fn get_images(
"date_desc" => "modified_at DESC NULLS LAST",
"size_asc" => "file_size ASC",
"size_desc" => "file_size DESC",
"rating_asc" => "rating ASC, modified_at DESC NULLS LAST",
"rating_desc" => "rating DESC, modified_at DESC NULLS LAST",
"duration_asc" => "duration_ms ASC NULLS LAST",
"duration_desc" => "duration_ms DESC NULLS LAST",
_ => "modified_at DESC NULLS LAST",
@@ -1126,15 +1343,17 @@ pub fn get_images(
"SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type,
media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error,
favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error,
generated_caption, caption_model, caption_updated_at, caption_error
generated_caption, caption_model, caption_updated_at, caption_error,
ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error
FROM images
WHERE (?1 IS NULL OR folder_id = ?1)
AND (?2 IS NULL OR filename LIKE ?2)
AND (?3 IS NULL OR media_kind = ?3)
AND (?4 = 0 OR favorite = 1)
AND (?5 = 0 OR embedding_status = 'failed')
AND rating >= ?5
AND (?6 = 0 OR embedding_status = 'failed')
ORDER BY {}
LIMIT ?6 OFFSET ?7",
LIMIT ?7 OFFSET ?8",
order
);
let mut stmt = conn.prepare(&sql)?;
@@ -1144,6 +1363,7 @@ pub fn get_images(
search_pattern,
media_kind,
favorites_flag,
rating_min,
embedding_failed_flag,
limit,
offset
@@ -1159,6 +1379,7 @@ pub fn count_images(
search: Option<&str>,
media_kind: Option<&str>,
favorites_only: bool,
rating_min: i64,
embedding_failed_only: bool,
) -> Result<i64> {
let search_pattern = search.map(|value| format!("%{}%", value));
@@ -1171,12 +1392,14 @@ pub fn count_images(
AND (?2 IS NULL OR filename LIKE ?2)
AND (?3 IS NULL OR media_kind = ?3)
AND (?4 = 0 OR favorite = 1)
AND (?5 = 0 OR embedding_status = 'failed')",
AND rating >= ?5
AND (?6 = 0 OR embedding_status = 'failed')",
params![
folder_id,
search_pattern,
media_kind,
favorites_flag,
rating_min,
embedding_failed_flag
],
|row| row.get(0),
@@ -1185,6 +1408,141 @@ pub fn count_images(
Ok(count)
}
pub fn search_images_by_tag(
conn: &Connection,
query: &str,
folder_id: Option<i64>,
media_kind: Option<&str>,
favorites_only: bool,
rating_min: i64,
limit: usize,
) -> Result<Vec<ImageRecord>> {
let normalized_query = query.trim().to_ascii_lowercase();
if normalized_query.is_empty() {
return Ok(Vec::new());
}
let favorites_flag = i64::from(favorites_only);
let mut stmt = conn.prepare(
"SELECT DISTINCT i.id
FROM images i
JOIN image_tags t ON t.image_id = i.id
WHERE (?1 IS NULL OR i.folder_id = ?1)
AND (?2 IS NULL OR i.media_kind = ?2)
AND (?3 = 0 OR i.favorite = 1)
AND i.rating >= ?4
AND LOWER(TRIM(t.tag)) = ?5
ORDER BY i.rating DESC, i.modified_at DESC NULLS LAST, i.filename ASC
LIMIT ?6",
)?;
let image_ids = stmt
.query_map(
params![
folder_id,
media_kind,
favorites_flag,
rating_min,
normalized_query,
limit as i64
],
|row| row.get::<_, i64>(0),
)?
.collect::<rusqlite::Result<Vec<_>>>()?;
get_images_by_ids(conn, &image_ids)
}
pub fn search_tags_autocomplete(
conn: &Connection,
query: &str,
folder_id: Option<i64>,
limit: usize,
) -> Result<Vec<ExploreTagEntry>> {
let pattern = format!("%{}%", query.to_lowercase());
let mut stmt = conn.prepare(
"SELECT t.tag, COUNT(DISTINCT t.image_id) AS tag_count, MIN(t.image_id) AS representative_image_id
FROM image_tags t
JOIN images i ON i.id = t.image_id
WHERE (?1 IS NULL OR i.folder_id = ?1)
AND LOWER(t.tag) LIKE ?2
GROUP BY t.tag
ORDER BY tag_count DESC, t.tag ASC
LIMIT ?3",
)?;
let rows = stmt
.query_map(params![folder_id, pattern, limit as i64], |row| {
Ok(ExploreTagEntry {
tag: row.get(0)?,
count: row.get(1)?,
representative_image_id: row.get(2)?,
thumbnail_path: None, // skip per-suggestion thumbnail for speed
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub struct ImagePathRecord {
pub id: i64,
pub path: String,
pub thumbnail_path: Option<String>,
}
pub fn get_all_image_paths(
conn: &Connection,
folder_id: Option<i64>,
) -> Result<Vec<ImagePathRecord>> {
let mut stmt = conn.prepare(
"SELECT id, path, thumbnail_path FROM images WHERE (?1 IS NULL OR folder_id = ?1) ORDER BY id",
)?;
let rows = stmt
.query_map(params![folder_id], |row| {
Ok(ImagePathRecord {
id: row.get(0)?,
path: row.get(1)?,
thumbnail_path: row.get(2)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub fn get_explore_tags(
conn: &Connection,
folder_id: Option<i64>,
limit: usize,
) -> Result<Vec<ExploreTagEntry>> {
let mut stmt = conn.prepare(
"SELECT t.tag, COUNT(DISTINCT t.image_id) AS tag_count, MIN(t.image_id) AS representative_image_id
FROM image_tags t
JOIN images i ON i.id = t.image_id
WHERE (?1 IS NULL OR i.folder_id = ?1)
GROUP BY t.tag
HAVING COUNT(DISTINCT t.image_id) >= 2
ORDER BY tag_count DESC, t.tag ASC
LIMIT ?2",
)?;
let rows = stmt
.query_map(params![folder_id, limit as i64], |row| {
let representative_image_id = row.get::<_, i64>(2)?;
let thumbnail_path = get_image_by_id(conn, representative_image_id)
.ok()
.and_then(|image| image.thumbnail_path);
Ok(ExploreTagEntry {
tag: row.get(0)?,
count: row.get(1)?,
representative_image_id,
thumbnail_path,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub fn get_failed_embedding_images(
conn: &Connection,
folder_id: i64,
@@ -1240,6 +1598,299 @@ pub fn mark_caption_failed(conn: &Connection, image_id: i64, error: &str) -> Res
Ok(())
}
// ---------------------------------------------------------------------------
// Tagging jobs
// ---------------------------------------------------------------------------
pub fn enqueue_tagging_job(conn: &Connection, image_id: i64) -> Result<()> {
conn.execute(
"INSERT INTO tagging_jobs (image_id, status, attempts, last_error, created_at, updated_at)
VALUES (?1, 'pending', 0, NULL, datetime('now'), datetime('now'))
ON CONFLICT(image_id) DO UPDATE SET
status = CASE WHEN status != 'processing' THEN 'pending' ELSE status END,
last_error = CASE WHEN status != 'processing' THEN NULL ELSE last_error END,
updated_at = datetime('now')",
[image_id],
)?;
Ok(())
}
pub fn claim_tagging_jobs(
conn: &mut Connection,
paused_folder_ids: &std::collections::HashSet<i64>,
limit: usize,
) -> Result<Vec<TaggingJob>> {
let tx = conn.transaction()?;
let candidates = get_pending_tagging_jobs_excluding(&tx, paused_folder_ids, limit * 2)?;
let mut claimed = Vec::new();
for job in candidates {
let updated = tx.execute(
"UPDATE tagging_jobs
SET status = 'processing', attempts = attempts + 1, updated_at = datetime('now')
WHERE image_id = ?1 AND status = 'pending'",
[job.image_id],
)?;
if updated > 0 {
claimed.push(job);
if claimed.len() >= limit {
break;
}
}
}
tx.commit()?;
Ok(claimed)
}
fn get_pending_tagging_jobs_excluding(
conn: &Connection,
paused_folder_ids: &std::collections::HashSet<i64>,
limit: usize,
) -> Result<Vec<TaggingJob>> {
let mut stmt = conn.prepare(
"SELECT j.image_id, i.folder_id, i.path
FROM tagging_jobs j
JOIN images i ON i.id = j.image_id
WHERE j.status = 'pending'
ORDER BY j.created_at ASC
LIMIT ?1",
)?;
let rows = stmt
.query_map([limit as i64], |row| {
Ok(TaggingJob {
image_id: row.get(0)?,
folder_id: row.get(1)?,
path: row.get(2)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows
.into_iter()
.filter(|job| !paused_folder_ids.contains(&job.folder_id))
.collect())
}
pub fn update_ai_tags(
conn: &Connection,
image_id: i64,
tags: &[(String, f64)],
rating: &str,
model: &str,
) -> Result<()> {
// NOTE: callers are responsible for wrapping this in a transaction.
// Do NOT open a nested transaction here — rusqlite/SQLite do not support
// nested transactions and will error with "cannot start a transaction
// within a transaction".
// Remove previous AI tags for this image before inserting fresh ones
conn.execute(
"DELETE FROM image_tags WHERE image_id = ?1 AND source = 'ai'",
[image_id],
)?;
let mut stmt = conn.prepare(
"INSERT INTO image_tags (image_id, tag, source, ai_model, confidence, created_at)
VALUES (?1, ?2, 'ai', ?3, ?4, datetime('now'))
ON CONFLICT(image_id, tag) DO UPDATE SET
source = 'ai',
ai_model = excluded.ai_model,
confidence = excluded.confidence",
)?;
for (tag, confidence) in tags {
stmt.execute(params![image_id, tag, model, confidence])?;
}
drop(stmt);
conn.execute(
"UPDATE images
SET ai_rating = ?2,
ai_tagger_model = ?3,
ai_tagged_at = datetime('now'),
ai_tagger_error = NULL
WHERE id = ?1",
params![image_id, rating, model],
)?;
conn.execute("DELETE FROM tagging_jobs WHERE image_id = ?1", [image_id])?;
Ok(())
}
pub fn mark_tagging_failed(conn: &Connection, image_id: i64, error: &str) -> Result<()> {
conn.execute(
"UPDATE images SET ai_tagger_error = ?2 WHERE id = ?1",
params![image_id, error],
)?;
conn.execute(
"UPDATE tagging_jobs
SET status = 'failed', last_error = ?2, updated_at = datetime('now')
WHERE image_id = ?1",
params![image_id, error],
)?;
Ok(())
}
pub fn clear_tagging_jobs(conn: &Connection, folder_id: Option<i64>) -> Result<usize> {
// Rows currently being processed by the worker must not be deleted mid-flight
// — the worker holds a reference and will write results back after inference.
// Mark them 'cancelled' so the worker discards the result instead of saving it,
// then delete every non-processing row. On the next poll the worker will find
// no pending work and the queue will appear empty.
let deleted = match folder_id {
Some(fid) => {
conn.execute(
"UPDATE tagging_jobs
SET status = 'cancelled', last_error = NULL, updated_at = datetime('now')
WHERE status = 'processing'
AND image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[fid],
)?;
let n = conn.execute(
"DELETE FROM tagging_jobs
WHERE status != 'processing'
AND image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[fid],
)?;
conn.execute(
"UPDATE images
SET ai_tagger_error = NULL
WHERE folder_id = ?1 AND ai_tagged_at IS NULL",
[fid],
)?;
n
}
None => {
conn.execute(
"UPDATE tagging_jobs
SET status = 'cancelled', last_error = NULL, updated_at = datetime('now')
WHERE status = 'processing'",
[],
)?;
let n = conn.execute("DELETE FROM tagging_jobs WHERE status != 'processing'", [])?;
conn.execute(
"UPDATE images SET ai_tagger_error = NULL WHERE ai_tagged_at IS NULL",
[],
)?;
n
}
};
Ok(deleted)
}
pub fn get_image_tags(conn: &Connection, image_id: i64) -> Result<Vec<ImageTag>> {
let mut stmt = conn.prepare(
"SELECT id, image_id, tag, source, ai_model, confidence, created_at
FROM image_tags
WHERE image_id = ?1
ORDER BY source DESC, confidence DESC NULLS LAST, tag ASC",
)?;
let rows = stmt
.query_map([image_id], |row| {
Ok(ImageTag {
id: row.get(0)?,
image_id: row.get(1)?,
tag: row.get(2)?,
source: row.get(3)?,
ai_model: row.get(4)?,
confidence: row.get(5)?,
created_at: row.get(6)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub fn add_user_tag(conn: &Connection, image_id: i64, tag: &str) -> Result<ImageTag> {
conn.execute(
"INSERT INTO image_tags (image_id, tag, source, ai_model, confidence, created_at)
VALUES (?1, ?2, 'user', NULL, NULL, datetime('now'))
ON CONFLICT(image_id, tag) DO NOTHING",
params![image_id, tag],
)?;
let row = conn.query_row(
"SELECT id, image_id, tag, source, ai_model, confidence, created_at
FROM image_tags WHERE image_id = ?1 AND tag = ?2",
params![image_id, tag],
|row| {
Ok(ImageTag {
id: row.get(0)?,
image_id: row.get(1)?,
tag: row.get(2)?,
source: row.get(3)?,
ai_model: row.get(4)?,
confidence: row.get(5)?,
created_at: row.get(6)?,
})
},
)?;
Ok(row)
}
pub fn remove_tag(conn: &Connection, tag_id: i64) -> Result<()> {
conn.execute("DELETE FROM image_tags WHERE id = ?1", [tag_id])?;
Ok(())
}
pub fn enqueue_missing_tagging_jobs_for_folder(conn: &Connection, folder_id: i64) -> Result<usize> {
let inserted = conn.execute(
"INSERT INTO tagging_jobs (image_id, status, attempts, last_error, created_at, updated_at)
SELECT id, 'pending', 0, NULL, datetime('now'), datetime('now')
FROM images
WHERE folder_id = ?1
AND media_kind = 'image'
AND ai_tagged_at IS NULL
ON CONFLICT(image_id) DO UPDATE SET
-- Only reset to 'pending' if the row is not currently being
-- processed or cancelled; leave 'processing' rows untouched so
-- the worker can complete them and clean up normally.
status = CASE WHEN status NOT IN ('processing', 'cancelled') THEN 'pending' ELSE status END,
last_error = CASE WHEN status != 'processing' THEN NULL ELSE last_error END,
updated_at = datetime('now')",
[folder_id],
)?;
conn.execute(
"UPDATE images
SET ai_tagger_error = NULL
WHERE folder_id = ?1
AND media_kind = 'image'
AND ai_tagged_at IS NULL",
[folder_id],
)?;
Ok(inserted)
}
pub fn requeue_processing_tagging_jobs_for_folder(conn: &Connection, folder_id: i64) -> Result<()> {
conn.execute(
"UPDATE tagging_jobs
SET status = 'pending', updated_at = datetime('now')
WHERE status = 'processing'
AND image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[folder_id],
)?;
Ok(())
}
/// Returns `true` when the job row for `image_id` currently has status = 'cancelled'.
/// Used by the worker to discard inference results for jobs that were cancelled
/// while inference was running.
pub fn is_tagging_job_cancelled(conn: &Connection, image_id: i64) -> Result<bool> {
let count: i64 = conn.query_row(
"SELECT COUNT(*) FROM tagging_jobs WHERE image_id = ?1 AND status = 'cancelled'",
[image_id],
|row| row.get(0),
)?;
Ok(count > 0)
}
pub fn requeue_tagging_jobs(conn: &Connection, image_ids: &[i64]) -> Result<()> {
for image_id in image_ids {
conn.execute(
"UPDATE tagging_jobs
SET status = 'pending', updated_at = datetime('now')
WHERE image_id = ?1 AND status = 'processing'",
[image_id],
)?;
}
Ok(())
}
pub fn suggest_tags_from_caption(
conn: &Connection,
image_id: i64,
@@ -1332,6 +1983,10 @@ fn map_image_row(row: &Row<'_>) -> rusqlite::Result<ImageRecord> {
caption_model: row.get(24)?,
caption_updated_at: row.get(25)?,
caption_error: row.get(26)?,
ai_rating: row.get(27)?,
ai_tagger_model: row.get(28)?,
ai_tagged_at: row.get(29)?,
ai_tagger_error: row.get(30)?,
})
}
@@ -1373,6 +2028,43 @@ pub fn set_tag_cloud_cache(
Ok(())
}
/// Returns (groups_json, scanned_at_unix) for the given folder scope, if present.
pub fn get_duplicate_scan_cache(
conn: &Connection,
folder_scope: &str,
) -> Result<Option<(String, i64)>> {
match conn.query_row(
"SELECT groups_json, scanned_at FROM duplicate_scan_cache WHERE folder_scope = ?1",
params![folder_scope],
|row| Ok((row.get::<_, String>(0)?, row.get::<_, i64>(1)?)),
) {
Ok(row) => Ok(Some(row)),
Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
Err(e) => Err(e.into()),
}
}
/// Upserts the duplicate scan cache for the given scope.
pub fn set_duplicate_scan_cache(
conn: &Connection,
folder_scope: &str,
groups_json: &str,
) -> Result<()> {
let scanned_at = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
.unwrap_or(0);
conn.execute(
"INSERT INTO duplicate_scan_cache (folder_scope, scanned_at, groups_json)
VALUES (?1, ?2, ?3)
ON CONFLICT(folder_scope) DO UPDATE SET
scanned_at = excluded.scanned_at,
groups_json = excluded.groups_json",
params![folder_scope, scanned_at, groups_json],
)?;
Ok(())
}
fn ensure_column(conn: &Connection, table: &str, column: &str, definition: &str) -> Result<()> {
let mut stmt = conn.prepare(&format!("PRAGMA table_info({})", table))?;
let mut rows = stmt.query([])?;
+45
View File
@@ -74,6 +74,51 @@ impl ClipImageEmbedder {
Ok(self.embed_images(&[path.to_path_buf()])?.remove(0))
}
/// Embed a cropped region of an image without writing a temp file to disk.
/// `crop_x`, `crop_y`, `crop_w`, `crop_h` are normalized 0.01.0 coordinates.
pub fn embed_image_crop(
&self,
path: &Path,
crop_x: f32,
crop_y: f32,
crop_w: f32,
crop_h: f32,
) -> Result<Vec<f32>> {
let img = image::ImageReader::open(path)?
.with_guessed_format()?
.decode()?;
let img_w = img.width() as f32;
let img_h = img.height() as f32;
let x = ((crop_x * img_w) as u32).min(img.width().saturating_sub(1));
let y = ((crop_y * img_h) as u32).min(img.height().saturating_sub(1));
let w = ((crop_w * img_w) as u32).max(1).min(img.width() - x);
let h = ((crop_h * img_h) as u32).max(1).min(img.height() - y);
let cropped = img.crop_imm(x, y, w, h);
let resized = cropped.resize_to_fill(
self.image_size as u32,
self.image_size as u32,
image::imageops::FilterType::Triangle,
);
let raw = resized.to_rgb8().into_raw();
let tensor = candle_core::Tensor::from_vec(
raw,
(self.image_size, self.image_size, 3),
&candle_core::Device::Cpu,
)?
.permute((2, 0, 1))?
.to_dtype(candle_core::DType::F32)?
.affine(2.0 / 255.0, -1.0)?;
let batch = tensor.unsqueeze(0)?.to_device(&self.device)?;
let features = self.model.get_image_features(&batch)?;
let normalized = candle_transformers::models::clip::div_l2_norm(&features)?;
Ok(normalized.get(0)?.flatten_all()?.to_vec1::<f32>()?)
}
pub fn embed_images(&self, paths: &[PathBuf]) -> Result<Vec<Vec<f32>>> {
let images = load_images(paths, self.image_size)?.to_device(&self.device)?;
let features = self.model.get_image_features(&images)?;
+129
View File
@@ -0,0 +1,129 @@
use crate::vector;
use anyhow::Result;
use hnsw_rs::prelude::{DistCosine, Hnsw, Neighbour};
use rusqlite::Connection;
use std::collections::HashMap;
use std::sync::{OnceLock, RwLock};
const HNSW_MAX_CONNECTIONS: usize = 24;
const HNSW_EF_CONSTRUCTION: usize = 300;
const HNSW_EF_SEARCH: usize = 96;
struct CachedHnswIndex {
revision: String,
image_ids_by_external: Vec<i64>,
external_by_image_id: HashMap<i64, usize>,
hnsw: Hnsw<'static, f32, DistCosine>,
}
static IMAGE_HNSW_INDEX: OnceLock<RwLock<Option<CachedHnswIndex>>> = OnceLock::new();
fn cache() -> &'static RwLock<Option<CachedHnswIndex>> {
IMAGE_HNSW_INDEX.get_or_init(|| RwLock::new(None))
}
fn build_index(conn: &Connection) -> Result<CachedHnswIndex> {
let embeddings = vector::get_all_image_embeddings_with_ids(conn, None)?;
let max_elements = embeddings.len().max(1);
let max_layer = 16.min((max_elements as f32).ln().trunc() as usize).max(1);
let mut hnsw = Hnsw::<f32, DistCosine>::new(
HNSW_MAX_CONNECTIONS,
max_elements,
max_layer,
HNSW_EF_CONSTRUCTION,
DistCosine {},
);
let image_ids_by_external = embeddings
.iter()
.map(|(image_id, _)| *image_id)
.collect::<Vec<_>>();
let external_by_image_id = image_ids_by_external
.iter()
.enumerate()
.map(|(external_id, image_id)| (*image_id, external_id))
.collect::<HashMap<_, _>>();
let data_with_id = embeddings
.iter()
.enumerate()
.map(|(external_id, (_, embedding))| (embedding, external_id))
.collect::<Vec<_>>();
hnsw.parallel_insert(&data_with_id);
hnsw.set_searching_mode(true);
Ok(CachedHnswIndex {
revision: vector::get_embedding_revision(conn)?,
image_ids_by_external,
external_by_image_id,
hnsw,
})
}
fn ensure_index(conn: &Connection) -> Result<()> {
let revision = vector::get_embedding_revision(conn)?;
{
let guard = cache().read().expect("hnsw cache poisoned");
if guard.as_ref().map(|cached| cached.revision.as_str()) == Some(revision.as_str()) {
return Ok(());
}
}
let next = build_index(conn)?;
let mut guard = cache().write().expect("hnsw cache poisoned");
*guard = Some(next);
Ok(())
}
pub fn find_similar_image_matches(
conn: &Connection,
image_id: i64,
folder_id: Option<i64>,
threshold: f32,
offset: usize,
limit: usize,
) -> Result<Vec<(i64, f32)>> {
ensure_index(conn)?;
let query_embedding = match vector::get_image_embedding(conn, image_id)? {
Some(embedding) => embedding,
None => return Ok(Vec::new()),
};
let guard = cache().read().expect("hnsw cache poisoned");
let Some(cached) = guard.as_ref() else {
return Ok(Vec::new());
};
let knbn = (offset + limit).max(limit).saturating_add(32);
let neighbours: Vec<Neighbour> = if let Some(folder_id) = folder_id {
let mut allowed_ids = vector::get_all_image_embeddings_with_ids(conn, Some(folder_id))?
.into_iter()
.filter_map(|(allowed_image_id, _)| {
cached.external_by_image_id.get(&allowed_image_id).copied()
})
.collect::<Vec<_>>();
allowed_ids.sort_unstable();
cached
.hnsw
.search_filter(&query_embedding, knbn, HNSW_EF_SEARCH, Some(&allowed_ids))
} else {
cached.hnsw.search(&query_embedding, knbn, HNSW_EF_SEARCH)
};
let matches = neighbours
.into_iter()
.filter_map(|neighbour| {
let image_id_match = cached.image_ids_by_external.get(neighbour.d_id).copied()?;
if image_id_match == image_id || neighbour.distance > threshold {
return None;
}
Some((image_id_match, neighbour.distance))
})
.skip(offset)
.take(limit)
.collect::<Vec<_>>();
Ok(matches)
}
+168 -1
View File
@@ -3,6 +3,7 @@ use crate::db::{self, DbPool, EmbeddingJob, FolderJobProgress, ImageRecord, Inde
use crate::embedder::{embedding_source_path, ClipImageEmbedder};
use crate::media::{probe_video_metadata, MediaTools};
use crate::storage::{detect_storage_profile, RuntimeAdaptiveProfile, StorageProfile};
use crate::tagger::{self, WdTagger};
use crate::thumbnail;
use crate::vector;
use anyhow::Result;
@@ -35,6 +36,7 @@ struct PausedWorkerFolders {
metadata: HashSet<i64>,
embedding: HashSet<i64>,
caption: HashSet<i64>,
tagging: HashSet<i64>,
}
#[derive(Clone, Copy)]
@@ -43,6 +45,7 @@ pub struct FolderWorkerPausedState {
pub metadata: bool,
pub embedding: bool,
pub caption: bool,
pub tagging: bool,
}
pub fn set_worker_paused(worker: &str, folder_id: i64, paused: bool) {
@@ -55,6 +58,7 @@ pub fn set_worker_paused(worker: &str, folder_id: i64, paused: bool) {
"metadata" => Some(&mut paused_folders.metadata),
"embedding" => Some(&mut paused_folders.embedding),
"caption" => Some(&mut paused_folders.caption),
"tagging" => Some(&mut paused_folders.tagging),
_ => None,
};
@@ -87,6 +91,7 @@ pub fn get_worker_paused_states(folder_ids: &[i64]) -> HashMap<i64, FolderWorker
metadata: paused_folders.metadata.contains(&folder_id),
embedding: paused_folders.embedding.contains(&folder_id),
caption: paused_folders.caption.contains(&folder_id),
tagging: paused_folders.tagging.contains(&folder_id),
},
)
})
@@ -105,6 +110,8 @@ fn paused_folder_ids(worker: &str) -> HashSet<i64> {
"thumbnail" => paused_folders.thumbnail.clone(),
"metadata" => paused_folders.metadata.clone(),
"embedding" => paused_folders.embedding.clone(),
"caption" => paused_folders.caption.clone(),
"tagging" => paused_folders.tagging.clone(),
_ => HashSet::new(),
}
}
@@ -191,6 +198,12 @@ pub fn start_caption_worker(app: AppHandle, pool: DbPool, app_data_dir: PathBuf)
let mut captioner: Option<FlorenceCaptioner> = None;
println!("Caption worker started.");
loop {
// If the acceleration setting changed, drop the cached session so
// the next batch picks it up with the new execution provider.
if captioner::CAPTION_SESSION_DIRTY.swap(false, std::sync::atomic::Ordering::Relaxed) {
println!("Caption worker: acceleration setting changed — resetting session.");
captioner = None;
}
if let Err(error) = process_caption_batch(&app, &pool, &app_data_dir, &mut captioner) {
eprintln!("Caption worker error: {}", error);
captioner = None;
@@ -200,6 +213,28 @@ pub fn start_caption_worker(app: AppHandle, pool: DbPool, app_data_dir: PathBuf)
});
}
pub fn start_tagging_worker(app: AppHandle, pool: DbPool, app_data_dir: PathBuf) {
std::thread::spawn(move || {
let mut tagger_instance: Option<WdTagger> = None;
println!("Tagging worker started.");
loop {
// If the acceleration setting changed, drop the cached session so
// the next batch picks it up with the new execution provider.
if tagger::TAGGER_SESSION_DIRTY.swap(false, std::sync::atomic::Ordering::Relaxed) {
println!("Tagging worker: acceleration setting changed — resetting session.");
tagger_instance = None;
}
if let Err(error) =
process_tagging_batch(&app, &pool, &app_data_dir, &mut tagger_instance)
{
eprintln!("Tagging worker error: {}", error);
tagger_instance = None;
}
std::thread::sleep(std::time::Duration::from_millis(750));
}
});
}
fn do_index(app: AppHandle, pool: DbPool, folder_id: i64, folder_path: PathBuf) -> Result<()> {
let existing_entries = {
let conn = pool.get()?;
@@ -296,6 +331,7 @@ fn do_index(app: AppHandle, pool: DbPool, folder_id: i64, folder_path: PathBuf)
if !missing_ids.is_empty() {
db::delete_images_by_ids(&conn, &missing_ids)?;
}
let _ = db::backfill_embedding_jobs(&conn)?;
db::update_folder_count(&conn, folder_id)?;
}
@@ -370,6 +406,10 @@ fn build_record(
caption_model: None,
caption_updated_at: None,
caption_error: None,
ai_rating: None,
ai_tagger_model: None,
ai_tagged_at: None,
ai_tagger_error: None,
})
}
@@ -825,6 +865,133 @@ fn process_caption_batch(
Ok(())
}
fn process_tagging_batch(
app: &AppHandle,
pool: &DbPool,
app_data_dir: &Path,
tagger_instance: &mut Option<WdTagger>,
) -> Result<()> {
if !tagger::tagger_model_status(app_data_dir).ready {
return Ok(());
}
let paused_folders = paused_folder_ids("tagging");
let batch_size = crate::tagger::tagger_batch_size(app_data_dir);
let jobs = with_db_write_lock(|| {
let mut conn = pool.get()?;
db::claim_tagging_jobs(&mut conn, &paused_folders, batch_size)
})?;
if jobs.is_empty() {
return Ok(());
}
if tagger_instance.is_none() {
match WdTagger::new(app_data_dir) {
Ok(model) => *tagger_instance = Some(model),
Err(error) => {
with_db_write_lock(|| {
let conn = pool.get()?;
db::requeue_tagging_jobs(
&conn,
&jobs.iter().map(|job| job.image_id).collect::<Vec<_>>(),
)
})?;
return Err(error);
}
}
}
let folder_ids = jobs.iter().map(|job| job.folder_id).collect::<HashSet<_>>();
emit_folder_job_progress(
app,
pool,
&folder_ids.iter().copied().collect::<Vec<_>>(),
false,
);
let tagger_ref = tagger_instance
.as_mut()
.expect("tagger should be initialized before tagging batch processing");
let tag_results = jobs
.iter()
.map(|job| {
(
job.clone(),
tagger_ref.run(Path::new(&job.path), tagger::DEFAULT_MAX_TAGS),
)
})
.collect::<Vec<_>>();
let updated_images = with_db_write_lock(|| {
let mut conn = pool.get()?;
let tx = conn.transaction()?;
let mut updated_images = Vec::with_capacity(tag_results.len());
for (job, tag_result) in &tag_results {
// If the job was cancelled while inference was running, discard
// the result and delete the row — don't save tags or mark failed.
if db::is_tagging_job_cancelled(&tx, job.image_id)? {
tx.execute(
"DELETE FROM tagging_jobs WHERE image_id = ?1",
[job.image_id],
)?;
continue;
}
match tag_result {
Ok(output) => {
let tag_pairs: Vec<(String, f64)> = output
.tags
.iter()
.map(|t| (t.tag.clone(), t.confidence as f64))
.collect();
db::update_ai_tags(
&tx,
job.image_id,
&tag_pairs,
&output.rating,
tagger::WD_TAGGER_MODEL_NAME,
)?;
}
Err(error) => {
db::mark_tagging_failed(&tx, job.image_id, &error.to_string())?;
}
}
updated_images.push(db::get_image_by_id(&tx, job.image_id)?);
}
tx.commit()?;
Ok(updated_images)
})
.or_else(|db_err| {
// The DB write failed. Try to requeue the claimed jobs so they aren't
// left stuck in 'processing' until the next app restart.
let image_ids: Vec<i64> = jobs.iter().map(|job| job.image_id).collect();
let _ = with_db_write_lock(|| {
let conn = pool.get()?;
db::requeue_tagging_jobs(&conn, &image_ids)
});
Err(db_err)
})?;
if !updated_images.is_empty() {
let folder_ids = updated_images
.iter()
.map(|image| image.folder_id)
.collect::<HashSet<_>>();
emit_media_updates(
app,
&MediaUpdateBatch {
images: updated_images,
},
);
emit_folder_job_progress(app, pool, &folder_ids.into_iter().collect::<Vec<_>>(), true);
}
Ok(())
}
fn active_indexing_folders() -> HashSet<i64> {
ACTIVE_INDEXING_FOLDERS
.get_or_init(|| Mutex::new(HashSet::new()))
@@ -910,7 +1077,7 @@ fn emit_media_updates(app: &AppHandle, batch: &MediaUpdateBatch) {
let _ = app.emit("media-updated", batch);
}
fn emit_folder_job_progress(app: &AppHandle, pool: &DbPool, folder_ids: &[i64], force: bool) {
pub fn emit_folder_job_progress(app: &AppHandle, pool: &DbPool, folder_ids: &[i64], force: bool) {
let mut unique_folder_ids = folder_ids.iter().copied().collect::<Vec<_>>();
unique_folder_ids.sort_unstable();
unique_folder_ids.dedup();
+11
View File
@@ -2,6 +2,7 @@ mod captioner;
mod commands;
mod db;
mod embedder;
mod hnsw_index;
mod indexer;
mod media;
mod storage;
@@ -86,9 +87,11 @@ pub fn run() {
commands::reindex_folder,
commands::update_image_details,
commands::find_similar_images,
commands::find_similar_by_region,
commands::debug_similar_images,
commands::retry_failed_embeddings,
commands::semantic_search_images,
commands::search_images_by_tag,
commands::get_caption_model_status,
commands::get_caption_acceleration,
commands::set_caption_acceleration,
@@ -107,6 +110,8 @@ pub fn run() {
commands::set_worker_paused,
commands::get_worker_states,
commands::get_tag_cloud,
commands::get_explore_tags,
commands::get_images_by_ids,
commands::get_failed_embedding_images,
commands::get_tagger_model_status,
commands::get_tagger_acceleration,
@@ -114,6 +119,8 @@ pub fn run() {
commands::probe_tagger_runtime,
commands::get_tagger_threshold,
commands::set_tagger_threshold,
commands::get_tagger_batch_size,
commands::set_tagger_batch_size,
commands::prepare_tagger_model,
commands::delete_tagger_model,
commands::queue_tagging_jobs,
@@ -121,6 +128,10 @@ pub fn run() {
commands::get_image_tags,
commands::add_user_tag,
commands::remove_tag,
commands::search_tags_autocomplete,
commands::find_duplicates,
commands::load_duplicate_scan_cache,
commands::delete_images_from_disk,
])
.run(tauri::generate_context!())
.expect("error while running tauri application");
+648
View File
@@ -0,0 +1,648 @@
use anyhow::Result;
use hf_hub::{api::sync::Api, Repo, RepoType};
use image::{imageops::FilterType, DynamicImage, ImageReader};
use ort::ep;
use ort::session::{builder::GraphOptimizationLevel, Session};
use ort::value::Tensor;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Instant;
pub const WD_TAGGER_MODEL_ID: &str = "SmilingWolf/wd-swinv2-tagger-v3";
pub const WD_TAGGER_MODEL_NAME: &str = "wd-swinv2-tagger-v3";
const TAGGER_ACCELERATION_FILE: &str = "settings/tagger_acceleration.txt";
const TAGGER_THRESHOLD_FILE: &str = "settings/tagger_threshold.txt";
const TAGGER_BATCH_SIZE_FILE: &str = "settings/tagger_batch_size.txt";
// Files required on disk before the tagger can run. The ONNX runtime DLLs
// are shared with the captioner and live in the same `onnxruntime/` directory.
const TAGGER_REQUIRED_FILES: &[&str] = &[
"onnxruntime/onnxruntime.dll",
"onnxruntime/onnxruntime_providers_shared.dll",
"onnxruntime/DirectML.dll",
"model.onnx",
"selected_tags.csv",
];
// Tags in these Danbooru categories are kept in the output.
// Category 0 = general, category 4 = character.
// Category 9 = rating (explicit/questionable/sensitive/general) used for
// `ai_rating` but NOT emitted as individual tags.
const GENERAL_CATEGORY: u8 = 0;
const CHARACTER_CATEGORY: u8 = 4;
const RATING_CATEGORY: u8 = 9;
pub const DEFAULT_THRESHOLD: f32 = 0.35;
pub const DEFAULT_MAX_TAGS: usize = 30;
/// Set to `true` by `set_tagger_acceleration` so the tagging worker loop
/// knows to drop its cached `WdTagger` and reload with the new EP.
pub static TAGGER_SESSION_DIRTY: AtomicBool = AtomicBool::new(false);
// ---------------------------------------------------------------------------
// Settings types
// ---------------------------------------------------------------------------
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum TaggerAcceleration {
Auto,
Cpu,
Directml,
}
impl TaggerAcceleration {
fn as_str(self) -> &'static str {
match self {
Self::Auto => "auto",
Self::Cpu => "cpu",
Self::Directml => "directml",
}
}
}
impl Default for TaggerAcceleration {
fn default() -> Self {
Self::Auto
}
}
// ---------------------------------------------------------------------------
// Status / probe types exposed to the frontend
// ---------------------------------------------------------------------------
#[derive(Serialize)]
pub struct TaggerModelStatus {
pub model_id: &'static str,
pub model_name: &'static str,
pub local_dir: String,
pub ready: bool,
pub missing_files: Vec<String>,
}
#[derive(Clone, Serialize)]
pub struct TaggerModelProgress {
pub total_files: usize,
pub completed_files: usize,
pub current_file: Option<String>,
pub done: bool,
}
// ---------------------------------------------------------------------------
// Runtime probe types exposed to the frontend
// ---------------------------------------------------------------------------
#[derive(Serialize)]
pub struct TaggerRuntimeProbe {
pub ready: bool,
pub acceleration: TaggerAcceleration,
pub session: TaggerRuntimeSessionProbe,
}
#[derive(Serialize)]
pub struct TaggerRuntimeSessionProbe {
pub file: &'static str,
pub inputs: Vec<String>,
pub outputs: Vec<String>,
}
// ---------------------------------------------------------------------------
// Tag record returned to callers
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, Serialize)]
pub struct TagResult {
pub tag: String,
pub confidence: f32,
}
#[derive(Debug, Clone, Serialize)]
pub struct TaggerOutput {
pub tags: Vec<TagResult>,
/// Highest-scoring rating label: "general" | "sensitive" | "questionable" | "explicit"
pub rating: String,
}
// ---------------------------------------------------------------------------
// Internal label table built from selected_tags.csv
// ---------------------------------------------------------------------------
#[derive(Debug, Clone)]
struct TagEntry {
name: String,
category: u8,
}
// ---------------------------------------------------------------------------
// Path helpers
// ---------------------------------------------------------------------------
pub fn model_dir(app_data_dir: &Path) -> PathBuf {
app_data_dir.join("models").join("wd-swinv2-tagger-v3")
}
// ---------------------------------------------------------------------------
// Settings persistence
// ---------------------------------------------------------------------------
pub fn tagger_acceleration(app_data_dir: &Path) -> TaggerAcceleration {
let path = app_data_dir.join(TAGGER_ACCELERATION_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return TaggerAcceleration::default();
};
match value.trim().to_ascii_lowercase().as_str() {
"cpu" => TaggerAcceleration::Cpu,
"directml" => TaggerAcceleration::Directml,
_ => TaggerAcceleration::Auto,
}
}
pub fn set_tagger_acceleration(
app_data_dir: &Path,
acceleration: TaggerAcceleration,
) -> Result<TaggerAcceleration> {
let path = app_data_dir.join(TAGGER_ACCELERATION_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, acceleration.as_str())?;
TAGGER_SESSION_DIRTY.store(true, Ordering::Relaxed);
Ok(acceleration)
}
pub fn tagger_threshold(app_data_dir: &Path) -> f32 {
let path = app_data_dir.join(TAGGER_THRESHOLD_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return DEFAULT_THRESHOLD;
};
value
.trim()
.parse::<f32>()
.unwrap_or(DEFAULT_THRESHOLD)
.clamp(0.01, 1.0)
}
pub fn set_tagger_threshold(app_data_dir: &Path, threshold: f32) -> Result<f32> {
let clamped = threshold.clamp(0.01, 1.0);
let path = app_data_dir.join(TAGGER_THRESHOLD_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, clamped.to_string())?;
Ok(clamped)
}
pub fn tagger_batch_size(app_data_dir: &Path) -> usize {
let path = app_data_dir.join(TAGGER_BATCH_SIZE_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return 8;
};
value
.trim()
.parse::<usize>()
.unwrap_or(8)
.clamp(1, 100)
}
pub fn set_tagger_batch_size(app_data_dir: &Path, batch_size: usize) -> Result<usize> {
let clamped = batch_size.clamp(1, 100);
let path = app_data_dir.join(TAGGER_BATCH_SIZE_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, clamped.to_string())?;
Ok(clamped)
}
// ---------------------------------------------------------------------------
// Model status / download
// ---------------------------------------------------------------------------
pub fn tagger_model_status(app_data_dir: &Path) -> TaggerModelStatus {
let local_dir = model_dir(app_data_dir);
// The ONNX runtime DLLs live in the caption model dir; reuse them.
let caption_model_dir = app_data_dir.join("models").join("florence-2-base-ft");
let missing_files = TAGGER_REQUIRED_FILES
.iter()
.filter(|file| {
let path = if file.starts_with("onnxruntime/") {
caption_model_dir.join(file)
} else {
local_dir.join(file)
};
!path.exists()
})
.map(|file| (*file).to_string())
.collect::<Vec<_>>();
TaggerModelStatus {
model_id: WD_TAGGER_MODEL_ID,
model_name: WD_TAGGER_MODEL_NAME,
local_dir: local_dir.to_string_lossy().to_string(),
ready: missing_files.is_empty(),
missing_files,
}
}
pub fn prepare_tagger_model_with_progress(
app_data_dir: &Path,
emit_progress: impl Fn(TaggerModelProgress),
) -> Result<TaggerModelStatus> {
let local_dir = model_dir(app_data_dir);
std::fs::create_dir_all(&local_dir)?;
// Only download the two tagger-specific files; ONNX runtime DLLs are
// already handled by the captioner download flow.
const DOWNLOAD_FILES: &[&str] = &["model.onnx", "selected_tags.csv"];
let api = Api::new()?;
let repo = api.repo(Repo::new(WD_TAGGER_MODEL_ID.to_string(), RepoType::Model));
let mut completed_files = DOWNLOAD_FILES
.iter()
.filter(|file| local_dir.join(file).exists())
.count();
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: None,
done: completed_files == DOWNLOAD_FILES.len(),
});
for file in DOWNLOAD_FILES {
let destination = local_dir.join(file);
if destination.exists() {
continue;
}
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: Some((*file).to_string()),
done: false,
});
let cached = repo.get(file)?;
std::fs::copy(cached, destination)?;
completed_files += 1;
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: Some((*file).to_string()),
done: completed_files == DOWNLOAD_FILES.len(),
});
}
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: None,
done: true,
});
Ok(tagger_model_status(app_data_dir))
}
pub fn delete_tagger_model(app_data_dir: &Path) -> Result<TaggerModelStatus> {
let local_dir = model_dir(app_data_dir);
if local_dir.exists() {
std::fs::remove_dir_all(&local_dir)?;
}
Ok(tagger_model_status(app_data_dir))
}
pub fn probe_tagger_runtime(app_data_dir: &Path) -> Result<TaggerRuntimeProbe> {
let status = tagger_model_status(app_data_dir);
if !status.ready {
anyhow::bail!(
"WD Tagger model is missing {} required file(s): {}",
status.missing_files.len(),
status.missing_files.join(", ")
);
}
let local_dir = model_dir(app_data_dir);
let caption_model_dir = app_data_dir.join("models").join("florence-2-base-ft");
crate::captioner::ensure_onnx_runtime(&caption_model_dir)?;
let acceleration = tagger_acceleration(app_data_dir);
let model_path = local_dir.join("model.onnx");
// Verify that the model file exists and has non-zero size before trying
// to create an ORT session (better error message on corruption).
let metadata = std::fs::metadata(&model_path)?;
if metadata.len() == 0 {
anyhow::bail!("model.onnx is empty");
}
// Actually create a session to verify the EP loads correctly.
let loaded_acceleration = match acceleration {
TaggerAcceleration::Cpu => {
create_tagger_session(&model_path, TaggerAcceleration::Cpu)?;
TaggerAcceleration::Cpu
}
TaggerAcceleration::Auto => {
// Try DirectML explicitly; if it fails the real session would have
// fallen back to CPU silently — report what would actually run.
let directml_ok =
create_tagger_session(&model_path, TaggerAcceleration::Directml).is_ok();
if directml_ok {
TaggerAcceleration::Directml
} else {
create_tagger_session(&model_path, TaggerAcceleration::Cpu)?;
TaggerAcceleration::Cpu
}
}
TaggerAcceleration::Directml => {
create_tagger_session(&model_path, TaggerAcceleration::Directml)?;
TaggerAcceleration::Directml
}
};
Ok(TaggerRuntimeProbe {
ready: true,
acceleration: loaded_acceleration,
session: TaggerRuntimeSessionProbe {
file: "model.onnx",
inputs: vec!["pixel_values".to_string()],
outputs: vec![format!("output [EP: {:?}]", loaded_acceleration)],
},
})
}
// ---------------------------------------------------------------------------
// Top-level inference entry point
// ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// Tagger implementation
// ---------------------------------------------------------------------------
pub struct WdTagger {
session: Session,
labels: Vec<TagEntry>,
threshold: f32,
input_size: usize,
}
impl WdTagger {
pub fn new(app_data_dir: &Path) -> Result<Self> {
let started_at = Instant::now();
let status = tagger_model_status(app_data_dir);
if !status.ready {
anyhow::bail!(
"WD tagger model is missing {} required file(s): {}",
status.missing_files.len(),
status.missing_files.join(", ")
);
}
let local_dir = model_dir(app_data_dir);
// The ONNX runtime DLLs are shared with the captioner; use the
// captioner's shared ORT init lock to avoid double-initialisation.
let caption_model_dir = app_data_dir.join("models").join("florence-2-base-ft");
crate::captioner::ensure_onnx_runtime(&caption_model_dir).map_err(|e| {
anyhow::anyhow!(
"ONNX Runtime not initialised — download the Florence-2 caption model first \
to get the shared runtime DLLs. Original error: {e}"
)
})?;
let acceleration = tagger_acceleration(app_data_dir);
let threshold = tagger_threshold(app_data_dir);
let model_path = local_dir.join("model.onnx");
let labels_path = local_dir.join("selected_tags.csv");
let session = create_tagger_session(&model_path, acceleration)?;
// Determine the input spatial size from the ONNX model graph.
// WD v3 models use (1, H, W, 3) where H == W, typically 448.
let input_size = {
let inputs = session.inputs();
let dim = inputs
.first()
.and_then(|inp| {
if let ort::value::ValueType::Tensor { shape, .. } = inp.dtype() {
shape
.get(1)
.and_then(|&d| if d > 0 { Some(d as usize) } else { None })
} else {
None
}
})
.unwrap_or(448);
dim
};
let labels = load_labels(&labels_path)?;
println!(
"WD tagger loaded in {:?} ({} labels, input {}x{}, {:?} acceleration)",
started_at.elapsed(),
labels.len(),
input_size,
input_size,
acceleration,
);
Ok(Self {
session,
labels,
threshold,
input_size,
})
}
pub fn run(&mut self, image_path: &Path, max_tags: usize) -> Result<TaggerOutput> {
let started_at = Instant::now();
let image_array = preprocess_image(image_path, self.input_size)?;
let batch_size = 1usize;
let input = Tensor::from_array((
[batch_size, self.input_size, self.input_size, 3usize],
image_array.into_boxed_slice(),
))
.map_err(|error| anyhow::anyhow!("{error}"))?;
let input_name: String = self.session.inputs()[0].name().to_string();
let outputs = self
.session
.run(ort::inputs! { input_name.as_str() => input })
.map_err(|error| anyhow::anyhow!("{error}"))?;
let (_, probabilities) = outputs[0]
.try_extract_tensor::<f32>()
.map_err(|error| anyhow::anyhow!("{error}"))?;
let probs: &[f32] = &probabilities;
if probs.len() != self.labels.len() {
anyhow::bail!(
"Model output length {} does not match label count {}",
probs.len(),
self.labels.len()
);
}
// Collect rating scores (category 9) - pick the argmax as the rating.
let rating = self
.labels
.iter()
.zip(probs.iter())
.filter(|(entry, _)| entry.category == RATING_CATEGORY)
.max_by(|(_, a), (_, b)| a.total_cmp(b))
.map(|(entry, _)| entry.name.clone())
.unwrap_or_else(|| "general".to_string());
// Collect general + character tags above threshold, sorted by confidence.
let mut tags: Vec<TagResult> = self
.labels
.iter()
.zip(probs.iter())
.filter(|(entry, prob)| {
(entry.category == GENERAL_CATEGORY || entry.category == CHARACTER_CATEGORY)
&& **prob >= self.threshold
})
.map(|(entry, prob)| TagResult {
tag: entry.name.clone(),
confidence: *prob,
})
.collect();
tags.sort_by(|a, b| b.confidence.total_cmp(&a.confidence));
tags.truncate(max_tags);
println!(
"WD tagger: {} tags (threshold={}, rating={}) in {:?} for {}",
tags.len(),
self.threshold,
rating,
started_at.elapsed(),
image_path.display(),
);
Ok(TaggerOutput { tags, rating })
}
}
// ---------------------------------------------------------------------------
// Session creation mirrors captioner's `create_session`
// ---------------------------------------------------------------------------
fn create_tagger_session(path: &Path, acceleration: TaggerAcceleration) -> Result<Session> {
let builder = Session::builder().map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_optimization_level(GraphOptimizationLevel::Level3)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let use_directml = matches!(
acceleration,
TaggerAcceleration::Auto | TaggerAcceleration::Directml
);
let builder = builder
.with_memory_pattern(!use_directml)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_parallel_execution(false)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_intra_threads(1)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut builder = match acceleration {
TaggerAcceleration::Cpu => {
println!("WD tagger: using CPU execution provider");
builder
}
TaggerAcceleration::Auto => builder
.with_execution_providers([ep::DirectML::default().build().fail_silently()])
.unwrap_or_else(|error| {
println!("WD tagger: DirectML unavailable, falling back to CPU");
error.recover()
}),
TaggerAcceleration::Directml => builder
.with_execution_providers([ep::DirectML::default().build().error_on_failure()])
.map_err(|error| anyhow::anyhow!("{error}"))?,
};
let session = builder
.commit_from_file(path)
.map_err(|error| anyhow::anyhow!("{error}"))?;
Ok(session)
}
// ---------------------------------------------------------------------------
// Label loading
// ---------------------------------------------------------------------------
fn load_labels(path: &Path) -> Result<Vec<TagEntry>> {
let mut reader = csv::Reader::from_path(path)?;
let mut entries = Vec::new();
for result in reader.records() {
let record = result?;
// CSV columns: tag_id, name, category, count
let name = record
.get(1)
.ok_or_else(|| anyhow::anyhow!("Missing name column in selected_tags.csv"))?
.replace('_', " ");
let category: u8 = record
.get(2)
.ok_or_else(|| anyhow::anyhow!("Missing category column in selected_tags.csv"))?
.parse()
.unwrap_or(0);
entries.push(TagEntry { name, category });
}
if entries.is_empty() {
anyhow::bail!("selected_tags.csv is empty or could not be parsed");
}
Ok(entries)
}
// ---------------------------------------------------------------------------
// Image preprocessing
// WD tagger expects: (1, H, W, 3) float32, raw [0,255] values, BGR channel
// order, padded to square with white (255,255,255).
// ---------------------------------------------------------------------------
fn preprocess_image(image_path: &Path, target_size: usize) -> Result<Vec<f32>> {
let image = ImageReader::open(image_path)?.decode()?;
// Composite any alpha channel onto a white background.
let image_rgba = image.to_rgba8();
let (width, height) = image_rgba.dimensions();
let mut canvas_rgba =
image::RgbaImage::from_pixel(width, height, image::Rgba([255, 255, 255, 255]));
image::imageops::overlay(&mut canvas_rgba, &image_rgba, 0, 0);
let image_rgb = DynamicImage::ImageRgba8(canvas_rgba).to_rgb8();
// Pad to square.
let max_dim = width.max(height);
let pad_left = (max_dim - width) / 2;
let pad_top = (max_dim - height) / 2;
let mut square = image::RgbImage::from_pixel(max_dim, max_dim, image::Rgb([255, 255, 255]));
image::imageops::overlay(&mut square, &image_rgb, pad_left as i64, pad_top as i64);
// Resize to model input size.
let resized = image::imageops::resize(
&square,
target_size as u32,
target_size as u32,
FilterType::CatmullRom,
);
// Flatten to (H, W, 3) float32 in BGR order, values in [0, 255].
let mut pixel_values = vec![0.0f32; target_size * target_size * 3];
for (x, y, pixel) in resized.enumerate_pixels() {
let base = (y as usize * target_size + x as usize) * 3;
// BGR order
pixel_values[base] = f32::from(pixel[2]); // B
pixel_values[base + 1] = f32::from(pixel[1]); // G
pixel_values[base + 2] = f32::from(pixel[0]); // R
}
Ok(pixel_values)
}
+160 -5
View File
@@ -80,7 +80,12 @@ pub fn upsert_caption_embedding(conn: &Connection, image_id: i64, embedding: &[f
Ok(())
}
pub fn find_similar_image_ids(conn: &Connection, image_id: i64, limit: usize) -> Result<Vec<i64>> {
pub fn find_similar_image_ids(
conn: &Connection,
image_id: i64,
limit: usize,
folder_id: Option<i64>,
) -> Result<Vec<i64>> {
let embedding: Vec<u8> = match conn.query_row(
"SELECT embedding FROM image_vec WHERE image_id = ?1",
[image_id],
@@ -91,19 +96,39 @@ pub fn find_similar_image_ids(conn: &Connection, image_id: i64, limit: usize) ->
Err(error) => return Err(error.into()),
};
if let Some(folder_id) = folder_id {
// Brute-force cosine scan scoped to the folder — avoids the KNN k=4096 limit
// and returns exact nearest neighbours within the folder.
let mut stmt = conn.prepare(
"SELECT v.image_id
FROM image_vec v
JOIN images i ON i.id = v.image_id
WHERE i.folder_id = ?2
AND v.image_id != ?3
ORDER BY vec_distance_cosine(v.embedding, vec_f32(?1)) ASC
LIMIT ?4",
)?;
let rows = stmt.query_map((&embedding, folder_id, image_id, limit as i64), |row| {
row.get::<_, i64>(0)
})?;
return Ok(rows.collect::<rusqlite::Result<Vec<_>>>()?);
}
// Global KNN search (no folder filter) — use the ANN index.
let mut stmt = conn.prepare(
"SELECT image_id
FROM image_vec
WHERE embedding MATCH vec_f32(?1)
AND k = ?2",
)?;
let rows = stmt.query_map((&embedding, (limit as i64) + 1), |row| row.get::<_, i64>(0))?;
let rows = stmt
.query_map((&embedding, (limit + 1) as i64), |row| row.get::<_, i64>(0))?
.collect::<rusqlite::Result<Vec<_>>>()?;
let mut ids = Vec::new();
for row in rows {
let candidate_id = row?;
if candidate_id != image_id {
ids.push(candidate_id);
if row != image_id {
ids.push(row);
}
if ids.len() >= limit {
break;
@@ -112,6 +137,102 @@ pub fn find_similar_image_ids(conn: &Connection, image_id: i64, limit: usize) ->
Ok(ids)
}
// pub fn find_similar_image_matches(
// conn: &Connection,
// image_id: i64,
// folder_id: Option<i64>,
// threshold: f32,
// offset: usize,
// limit: usize,
// ) -> Result<Vec<(i64, f32)>> {
// let embedding: Vec<u8> = match conn.query_row(
// "SELECT embedding FROM image_vec WHERE image_id = ?1",
// [image_id],
// |row| row.get(0),
// ) {
// Ok(embedding) => embedding,
// Err(SqliteError::QueryReturnedNoRows) => return Ok(Vec::new()),
// Err(error) => return Err(error.into()),
// };
// let query = match folder_id {
// Some(_) => {
// "SELECT v.image_id, vec_distance_cosine(v.embedding, vec_f32(?1)) AS distance
// FROM image_vec v
// JOIN images i ON i.id = v.image_id
// WHERE i.folder_id = ?2
// AND v.image_id != ?3
// AND vec_distance_cosine(v.embedding, vec_f32(?1)) <= ?4
// ORDER BY distance ASC
// LIMIT ?5 OFFSET ?6"
// }
// None => {
// "SELECT v.image_id, vec_distance_cosine(v.embedding, vec_f32(?1)) AS distance
// FROM image_vec v
// WHERE v.image_id != ?2
// AND vec_distance_cosine(v.embedding, vec_f32(?1)) <= ?3
// ORDER BY distance ASC
// LIMIT ?4 OFFSET ?5"
// }
// };
// let mut stmt = conn.prepare(query)?;
// match folder_id {
// Some(folder_id) => Ok(stmt
// .query_map(
// (
// &embedding,
// folder_id,
// image_id,
// threshold,
// limit as i64,
// offset as i64,
// ),
// |row| Ok((row.get::<_, i64>(0)?, row.get::<_, f32>(1)?)),
// )?
// .collect::<rusqlite::Result<Vec<_>>>()?),
// None => Ok(stmt
// .query_map(
// (&embedding, image_id, threshold, limit as i64, offset as i64),
// |row| Ok((row.get::<_, i64>(0)?, row.get::<_, f32>(1)?)),
// )?
// .collect::<rusqlite::Result<Vec<_>>>()?),
// }
// }
pub fn get_image_embedding(conn: &Connection, image_id: i64) -> Result<Option<Vec<f32>>> {
let embedding: Result<Vec<u8>, rusqlite::Error> = conn.query_row(
"SELECT embedding FROM image_vec WHERE image_id = ?1",
[image_id],
|row| row.get(0),
);
match embedding {
Ok(bytes) => Ok(Some(unpack_f32(&bytes))),
Err(SqliteError::QueryReturnedNoRows) => Ok(None),
Err(error) => Err(error.into()),
}
}
pub fn get_embedding_revision(conn: &Connection) -> Result<String> {
let count: i64 = conn.query_row("SELECT COUNT(*) FROM image_vec", [], |row| row.get(0))?;
let max_updated_at: Option<String> = conn.query_row(
"SELECT MAX(embedding_updated_at) FROM images WHERE embedding_status = 'ready'",
[],
|row| row.get(0),
)?;
Ok(format!("{}:{}", count, max_updated_at.unwrap_or_default()))
}
// fn image_ids_for_folder(
// conn: &Connection,
// folder_id: i64,
// ) -> Result<std::collections::HashSet<i64>> {
// let mut stmt = conn.prepare("SELECT id FROM images WHERE folder_id = ?1")?;
// let rows = stmt.query_map([folder_id], |row| row.get::<_, i64>(0))?;
// Ok(rows.collect::<rusqlite::Result<std::collections::HashSet<_>>>()?)
// }
/// Returns all stored image embeddings with their image IDs, optionally filtered to one folder.
/// Each entry is `(image_id, normalized_f32_embedding)`.
pub fn get_all_image_embeddings_with_ids(
@@ -189,6 +310,40 @@ pub fn search_image_ids_by_embedding(
Ok(ids)
}
/// Brute-force cosine search scoped to a single folder, ordered by ascending distance.
/// Used for region-based similarity search where we want folder-scoped results.
pub fn search_image_ids_by_embedding_in_folder(
conn: &Connection,
embedding: &[f32],
folder_id: i64,
exclude_image_id: Option<i64>,
limit: usize,
) -> Result<Vec<i64>> {
if embedding.len() != CLIP_VECTOR_DIM {
return Err(anyhow!(
"expected {}-dimensional embedding, got {}",
CLIP_VECTOR_DIM,
embedding.len()
));
}
let packed = pack_f32(embedding);
let exclude_id = exclude_image_id.unwrap_or(-1);
let mut stmt = conn.prepare(
"SELECT v.image_id
FROM image_vec v
JOIN images i ON i.id = v.image_id
WHERE i.folder_id = ?2
AND v.image_id != ?3
ORDER BY vec_distance_cosine(v.embedding, vec_f32(?1)) ASC
LIMIT ?4",
)?;
let rows = stmt.query_map((&packed, folder_id, exclude_id, limit as i64), |row| {
row.get::<_, i64>(0)
})?;
Ok(rows.collect::<rusqlite::Result<Vec<_>>>()?)
}
#[allow(dead_code)]
pub fn search_caption_ids_by_embedding(
conn: &Connection,
+5 -3
View File
@@ -4,9 +4,9 @@
"version": "0.1.0",
"identifier": "wtf.jezz.phokus",
"build": {
"beforeDevCommand": "pnpm dev",
"beforeDevCommand": "pnpm dev:vite",
"devUrl": "http://localhost:1420",
"beforeBuildCommand": "pnpm build",
"beforeBuildCommand": "pnpm build:vite",
"frontendDist": "../dist"
},
"app": {
@@ -25,7 +25,9 @@
"csp": null,
"assetProtocol": {
"enable": true,
"scope": ["**"]
"scope": [
"**"
]
}
}
},
+8
View File
@@ -6,6 +6,7 @@ import { Toolbar } from "./components/Toolbar";
import { Gallery } from "./components/Gallery";
import { Lightbox } from "./components/Lightbox";
import { TagCloud } from "./components/TagCloud";
import { DuplicateFinder } from "./components/DuplicateFinder";
import { TitleBar } from "./components/TitleBar";
import { SettingsModal } from "./components/SettingsModal";
@@ -14,6 +15,7 @@ export default function App() {
const loadBackgroundJobProgress = useGalleryStore((state) => state.loadBackgroundJobProgress);
const loadImages = useGalleryStore((state) => state.loadImages);
const loadCaptionModelStatus = useGalleryStore((state) => state.loadCaptionModelStatus);
const loadDuplicateScanCache = useGalleryStore((state) => state.loadDuplicateScanCache);
const subscribeToProgress = useGalleryStore((state) => state.subscribeToProgress);
const activeView = useGalleryStore((state) => state.activeView);
@@ -21,6 +23,7 @@ export default function App() {
loadFolders().then(() => {
void loadBackgroundJobProgress();
void loadCaptionModelStatus();
void loadDuplicateScanCache();
return loadImages(true);
});
let unlisten: (() => void) | undefined;
@@ -46,6 +49,11 @@ export default function App() {
<BackgroundTasks />
<TagCloud />
</>
) : activeView === "duplicates" ? (
<>
<BackgroundTasks />
<DuplicateFinder />
</>
) : (
<>
<Toolbar />
+39 -7
View File
@@ -58,6 +58,8 @@ export function BackgroundTasks() {
const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress);
const retryFailedEmbeddings = useGalleryStore((state) => state.retryFailedEmbeddings);
const clearTaggingJobs = useGalleryStore((state) => state.clearTaggingJobs);
const duplicateScanning = useGalleryStore((state) => state.duplicateScanning);
const duplicateScanProgress = useGalleryStore((state) => state.duplicateScanProgress);
const [expanded, setExpanded] = useState(false);
const [dismissed, setDismissed] = useState<Record<number, string>>({});
const [paused, setPaused] = useState<Record<number, Record<WorkerKey, boolean>>>({});
@@ -127,6 +129,7 @@ export function BackgroundTasks() {
};
const dismissTask = (id: number, snapshot: string) => {
if (id < 0) return; // system tasks (duplicate scan) cannot be dismissed
void clearTaggingJobs(id);
setDismissed((prev) => ({ ...prev, [id]: snapshot }));
setExpanded(false);
@@ -244,10 +247,35 @@ export function BackgroundTasks() {
.filter((t) => dismissed[t.id] !== t.snapshot);
}, [folders, indexingProgress, mediaJobProgress, dismissed]);
if (tasks.length === 0) return null;
// Synthetic task for duplicate scanning — negative id so dismiss/retry are suppressed
const duplicateScanTask: Task | null = duplicateScanning ? {
id: -1,
name: "Duplicate Scan",
stages: [{
label: "Hashing",
detail: duplicateScanProgress
? `${duplicateScanProgress.scanned.toLocaleString()} / ${duplicateScanProgress.total.toLocaleString()}`
: "Starting…",
progress: duplicateScanProgress && duplicateScanProgress.total > 0
? (duplicateScanProgress.scanned / duplicateScanProgress.total) * 100
: null,
failed: false,
}],
hasFailedEmbeddings: false,
hasFailedTagging: false,
pendingMediaWork: 1,
embeddingProcessed: 0,
embeddingTotal: 0,
currentFile: null,
snapshot: "",
} : null;
const primary = tasks[0];
const extraCount = tasks.length - 1;
const allTasks = duplicateScanTask ? [duplicateScanTask, ...tasks] : tasks;
if (allTasks.length === 0) return null;
const primary = allTasks[0];
const extraCount = allTasks.length - 1;
const hasFailed = tasks.some((t) => (t.hasFailedEmbeddings || t.hasFailedTagging) && t.pendingMediaWork === 0);
// Best progress bar value: use embedding progress if available (most informative),
@@ -348,8 +376,8 @@ export function BackgroundTasks() {
</button>
)}
{/* Expand chevron (only when multiple folders) */}
{tasks.length > 1 && (
{/* Expand chevron (only when multiple tasks) */}
{allTasks.length > 1 && (
<svg
className={`h-3.5 w-3.5 text-gray-600 transition-transform duration-200 shrink-0 ${expanded ? "rotate-180" : ""}`}
fill="none" viewBox="0 0 24 24" stroke="currentColor"
@@ -358,7 +386,8 @@ export function BackgroundTasks() {
</svg>
)}
{/* Dismiss */}
{/* Dismiss — hidden for system tasks like duplicate scan */}
{primary.id >= 0 && (
<button
className="p-1 rounded-md text-gray-600 hover:text-gray-300 hover:bg-white/8 transition-colors shrink-0"
title="Dismiss"
@@ -368,12 +397,13 @@ export function BackgroundTasks() {
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
)}
</div>
{/* Expanded panel — one row per folder */}
{expanded && (
<div className="border-t border-white/[0.06] bg-white/[0.02] px-5 py-3 space-y-3">
{tasks.map((task) => {
{allTasks.map((task) => {
const taskEmbeddingStage = task.stages.find((s) => s.label === "Embeddings");
const taskTaggingStage = task.stages.find((s) => s.label === "Tags");
const taskScanningStage = task.stages.find((s) => s.label === "Scanning");
@@ -453,6 +483,7 @@ export function BackgroundTasks() {
</button>
)}
{task.id >= 0 && (
<button
className="p-1 rounded-md text-gray-600 hover:text-gray-300 hover:bg-white/8 transition-colors shrink-0"
title="Dismiss"
@@ -462,6 +493,7 @@ export function BackgroundTasks() {
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
)}
</div>
{task.currentFile && (
+274
View File
@@ -0,0 +1,274 @@
import { useState } from "react";
import { convertFileSrc } from "@tauri-apps/api/core";
import { DuplicateGroup, useGalleryStore } from "../store";
function formatBytes(bytes: number): string {
if (bytes >= 1_073_741_824) return `${(bytes / 1_073_741_824).toFixed(1)} GB`;
if (bytes >= 1_048_576) return `${(bytes / 1_048_576).toFixed(1)} MB`;
if (bytes >= 1024) return `${(bytes / 1024).toFixed(0)} KB`;
return `${bytes} B`;
}
function DuplicateGroupCard({ group }: { group: DuplicateGroup }) {
const selectedIds = useGalleryStore((state) => state.duplicateSelectedIds);
const toggleDuplicateSelected = useGalleryStore((state) => state.toggleDuplicateSelected);
const selectAllDuplicates = useGalleryStore((state) => state.selectAllDuplicates);
const groupSelectedCount = group.images.filter((img) => selectedIds.has(img.id)).length;
const noneSelected = groupSelectedCount === 0;
// "Keep all but the first" — a common quick action
const handleKeepFirst = () => {
const toDelete = group.images.slice(1).map((img) => img.id);
// Clear any selection for this group first, then add the ones to delete
for (const img of group.images) {
if (selectedIds.has(img.id)) toggleDuplicateSelected(img.id);
}
selectAllDuplicates(toDelete);
};
return (
<div className="rounded-2xl border border-white/[0.07] bg-white/[0.02] p-4">
{/* Group header */}
<div className="mb-3 flex items-center justify-between gap-3">
<div className="flex items-center gap-2">
<span className="rounded-md border border-white/10 bg-white/[0.04] px-2 py-0.5 text-[11px] text-gray-400">
{group.images.length} copies
</span>
<span className="text-[11px] text-white/30">{formatBytes(group.file_size)} each</span>
<span className="text-[11px] text-white/20">
{formatBytes(group.file_size * (group.images.length - 1))} wasted
</span>
</div>
<div className="flex items-center gap-2">
{noneSelected ? (
<button
className="text-[11px] text-white/35 transition-colors hover:text-white/70"
onClick={handleKeepFirst}
>
Keep first
</button>
) : (
<button
className="text-[11px] text-white/35 transition-colors hover:text-white/70"
onClick={() => {
for (const img of group.images) {
if (selectedIds.has(img.id)) toggleDuplicateSelected(img.id);
}
}}
>
Deselect all
</button>
)}
</div>
</div>
{/* Image grid */}
<div className="flex flex-wrap gap-3">
{group.images.map((image) => {
const isSelected = selectedIds.has(image.id);
const src = image.thumbnail_path ? convertFileSrc(image.thumbnail_path) : null;
return (
<button
key={image.id}
className={`group relative overflow-hidden rounded-xl border transition-all ${
isSelected
? "border-red-400/50 ring-1 ring-red-400/30"
: "border-white/8 hover:border-white/20"
}`}
style={{ width: 140, height: 105 }}
onClick={() => toggleDuplicateSelected(image.id)}
title={image.path}
>
{src ? (
<img src={src} alt="" className="h-full w-full object-cover" draggable={false} />
) : (
<div className="h-full w-full bg-white/[0.03]" />
)}
{/* Delete overlay */}
{isSelected ? (
<div className="absolute inset-0 flex items-center justify-center bg-red-950/60">
<svg className="h-6 w-6 text-red-300" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M19 7l-.867 12.142A2 2 0 0116.138 21H7.862a2 2 0 01-1.995-1.858L5 7m5 4v6m4-6v6m1-10V4a1 1 0 00-1-1h-4a1 1 0 00-1 1v3M4 7h16" />
</svg>
</div>
) : null}
{/* Path tooltip on hover */}
<div className="absolute inset-x-0 bottom-0 bg-gradient-to-t from-black/90 to-transparent px-2 pb-1.5 pt-4 opacity-0 transition-opacity group-hover:opacity-100">
<p className="truncate text-[9px] text-white/60">{image.path.split(/[\\/]/).slice(-2).join("/")}</p>
</div>
</button>
);
})}
</div>
</div>
);
}
function formatRelativeTime(unixSecs: number): string {
const diff = Math.floor(Date.now() / 1000) - unixSecs;
if (diff < 60) return "just now";
if (diff < 3600) return `${Math.floor(diff / 60)}m ago`;
if (diff < 86400) return `${Math.floor(diff / 3600)}h ago`;
return `${Math.floor(diff / 86400)}d ago`;
}
export function DuplicateFinder() {
const duplicateGroups = useGalleryStore((state) => state.duplicateGroups);
const duplicateScanning = useGalleryStore((state) => state.duplicateScanning);
const duplicateScanProgress = useGalleryStore((state) => state.duplicateScanProgress);
const duplicateSelectedIds = useGalleryStore((state) => state.duplicateSelectedIds);
const duplicateLastScanned = useGalleryStore((state) => state.duplicateLastScanned);
const selectedFolderId = useGalleryStore((state) => state.selectedFolderId);
const scanDuplicates = useGalleryStore((state) => state.scanDuplicates);
const clearDuplicateSelection = useGalleryStore((state) => state.clearDuplicateSelection);
const selectKeepFirstAllGroups = useGalleryStore((state) => state.selectKeepFirstAllGroups);
const deleteSelectedDuplicates = useGalleryStore((state) => state.deleteSelectedDuplicates);
const [deleting, setDeleting] = useState(false);
const [deleteResult, setDeleteResult] = useState<string | null>(null);
const selectedCount = duplicateSelectedIds.size;
const hasResults = duplicateGroups.length > 0;
const hasScanned = hasResults || duplicateLastScanned !== null || (!duplicateScanning && duplicateScanProgress !== null);
const totalWasted = duplicateGroups.reduce(
(sum, g) => sum + g.file_size * (g.images.length - 1),
0,
);
const totalDuplicateImages = duplicateGroups.reduce((sum, g) => sum + g.images.length - 1, 0);
const handleDelete = async () => {
setDeleting(true);
setDeleteResult(null);
try {
const deleted = await deleteSelectedDuplicates();
setDeleteResult(`Deleted ${deleted} file${deleted === 1 ? "" : "s"}.`);
} catch (e) {
setDeleteResult(String(e));
} finally {
setDeleting(false);
}
};
const progressPercent =
duplicateScanProgress && duplicateScanProgress.total > 0
? Math.round((duplicateScanProgress.scanned / duplicateScanProgress.total) * 100)
: 0;
return (
<div className="flex min-h-0 flex-1 flex-col overflow-hidden bg-[#07080f]">
{/* Header */}
<div className="shrink-0 border-b border-white/[0.05] px-6 py-4">
<div className="flex items-center justify-between gap-4">
<div>
<h2 className="text-[15px] font-semibold text-white">Duplicate Finder</h2>
<p className="mt-0.5 text-[11px] text-white/30">
{duplicateScanning
? duplicateScanProgress
? `Scanning… ${duplicateScanProgress.scanned.toLocaleString()} / ${duplicateScanProgress.total.toLocaleString()}`
: "Starting scan…"
: hasResults
? `${duplicateGroups.length} group${duplicateGroups.length === 1 ? "" : "s"} · ${formatBytes(totalWasted)} reclaimable`
: duplicateLastScanned !== null
? "No duplicates found"
: "Scan your library for identical files"}
</p>
{!duplicateScanning && duplicateLastScanned !== null && (
<p className="mt-0.5 text-[10px] text-white/20">
Last scanned {formatRelativeTime(duplicateLastScanned)}
</p>
)}
</div>
<div className="flex items-center gap-2">
{/* Batch select — only shown when there are groups and nothing is selected yet */}
{hasResults && selectedCount === 0 && !deleting && (
<button
className="rounded-lg border border-white/10 bg-white/[0.04] px-3 py-1.5 text-xs text-gray-400 transition-colors hover:bg-white/[0.07] hover:text-white"
onClick={selectKeepFirstAllGroups}
title={`Mark ${totalDuplicateImages} duplicate${totalDuplicateImages === 1 ? "" : "s"} for deletion across all groups (keeps first in each)`}
>
Select all duplicates
</button>
)}
{selectedCount > 0 ? (
<>
<span className="text-[11px] text-white/40">{selectedCount} marked for deletion</span>
<button
className="rounded-lg border border-white/10 bg-white/[0.04] px-3 py-1.5 text-xs text-gray-400 transition-colors hover:bg-white/[0.07] hover:text-white disabled:opacity-40"
onClick={clearDuplicateSelection}
disabled={deleting}
>
Deselect all
</button>
<button
className="rounded-lg border border-red-400/25 bg-red-500/10 px-3 py-1.5 text-xs text-red-300 transition-colors hover:bg-red-500/15 disabled:cursor-not-allowed disabled:opacity-40"
onClick={handleDelete}
disabled={deleting}
>
{deleting ? "Deleting…" : `Delete ${selectedCount} file${selectedCount === 1 ? "" : "s"}`}
</button>
</>
) : null}
<button
className="rounded-lg border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-40"
onClick={() => { setDeleteResult(null); void scanDuplicates(selectedFolderId); }}
disabled={duplicateScanning}
>
{duplicateScanning ? "Scanning…" : hasScanned ? "Rescan" : "Scan for duplicates"}
</button>
</div>
</div>
{/* Progress bar */}
{duplicateScanning && duplicateScanProgress ? (
<div className="mt-3 h-px overflow-hidden rounded-full bg-white/[0.07]">
<div
className="h-full rounded-full bg-blue-500/60 transition-[width] duration-200"
style={{ width: `${progressPercent}%` }}
/>
</div>
) : null}
{deleteResult ? (
<p className="mt-2 text-[11px] text-white/40">{deleteResult}</p>
) : null}
</div>
{/* Body */}
{duplicateScanning && !hasResults ? (
<div className="flex flex-1 items-center justify-center gap-3 text-white/25">
<div className="h-5 w-5 animate-spin rounded-full border-2 border-white/15 border-t-white/50" />
<span className="text-sm">Hashing files</span>
</div>
) : !hasScanned ? (
<div className="flex flex-1 items-center justify-center px-8">
<div className="max-w-sm text-center">
<svg className="mx-auto mb-4 h-10 w-10 text-white/10" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1}
d="M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z" />
</svg>
<p className="text-sm text-white/30">
Finds files with identical content regardless of filename or location.
Click <strong className="text-white/50">Scan for duplicates</strong> to begin.
</p>
<p className="mt-2 text-xs text-white/20">
Large libraries may take a minute files are hashed from disk.
</p>
</div>
</div>
) : duplicateGroups.length === 0 ? (
<div className="flex flex-1 items-center justify-center">
<p className="text-sm text-white/25">No duplicate files found.</p>
</div>
) : (
<div className="overflow-y-auto px-6 py-5">
<div className="space-y-4">
{duplicateGroups.map((group) => (
<DuplicateGroupCard key={group.file_hash} group={group} />
))}
</div>
</div>
)}
</div>
);
}
+37 -23
View File
@@ -1,6 +1,6 @@
import { useEffect, useRef, useCallback, useState } from "react";
import { convertFileSrc } from "@tauri-apps/api/core";
import { ImageRecord, tileSizeForZoom, useGalleryStore } from "../store";
import { ImageRecord, parseSearchValue, tileSizeForZoom, useGalleryStore } from "../store";
const GAP = 6;
@@ -30,6 +30,7 @@ function ContextMenu({
const openImage = useGalleryStore((state) => state.openImage);
const updateImageDetails = useGalleryStore((state) => state.updateImageDetails);
const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages);
const similarScope = useGalleryStore((state) => state.similarScope);
const canFindSimilar = image.embedding_status === "ready";
return (
@@ -59,7 +60,7 @@ function ContextMenu({
}`}
onClick={async () => {
if (!canFindSimilar) return;
await loadSimilarImages(image.id);
await loadSimilarImages(image.id, similarScope === "current_folder" ? image.folder_id : null, true, image.folder_id);
onClose();
}}
disabled={!canFindSimilar}
@@ -115,6 +116,7 @@ function ImageTile({
const [loaded, setLoaded] = useState(false);
const [errored, setErrored] = useState(false);
const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages);
const similarScope = useGalleryStore((state) => state.similarScope);
const canFindSimilar = image.embedding_status === "ready";
const src = image.thumbnail_path
@@ -181,6 +183,15 @@ function ImageTile({
</svg>
</div>
)}
{image.rating > 0 && (
<div className="flex items-center gap-0.5 rounded-md bg-black/60 px-1.5 py-1 text-amber-300 backdrop-blur-sm">
{Array.from({ length: image.rating }, (_, index) => (
<svg key={index} className="h-2.5 w-2.5" fill="currentColor" viewBox="0 0 20 20">
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.176 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81H7.03a1 1 0 00.951-.69l1.07-3.292z" />
</svg>
))}
</div>
)}
{image.media_kind === "video" && image.duration_ms && (
<div className="rounded-md bg-black/60 px-1.5 py-0.5 text-[10px] font-medium text-white/80 backdrop-blur-sm">
{formatDuration(image.duration_ms)}
@@ -230,7 +241,7 @@ function ImageTile({
onClick={(event) => {
event.stopPropagation();
if (!canFindSimilar) return;
void loadSimilarImages(image.id);
void loadSimilarImages(image.id, similarScope === "current_folder" ? image.folder_id : null, true, image.folder_id);
}}
disabled={!canFindSimilar}
>
@@ -250,11 +261,11 @@ export function Gallery() {
const loadingImages = useGalleryStore((state) => state.loadingImages);
const zoomPreset = useGalleryStore((state) => state.zoomPreset);
const search = useGalleryStore((state) => state.search);
const searchMode = useGalleryStore((state) => state.searchMode);
const collectionTitle = useGalleryStore((state) => state.collectionTitle);
const imageLoadError = useGalleryStore((state) => state.imageLoadError);
const galleryScrollResetKey = useGalleryStore((state) => state.galleryScrollResetKey);
const isSimilarResults = collectionTitle === "Similar Images";
const parsedSearch = parseSearchValue(search);
const parentRef = useRef<HTMLDivElement>(null);
const [contextMenu, setContextMenu] = useState<{ x: number; y: number; image: ImageRecord } | null>(null);
@@ -262,6 +273,7 @@ export function Gallery() {
const handleScroll = useCallback(() => {
const element = parentRef.current;
if (!element) return;
if (element.scrollTop < 24) return;
const nearBottom = element.scrollTop + element.clientHeight >= element.scrollHeight - 600;
if (nearBottom && !loadingImages && images.length < totalImages) {
void loadMoreImages();
@@ -295,33 +307,34 @@ export function Gallery() {
};
}, []);
if (images.length === 0 && loadingImages) {
return (
<div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8">
<div ref={parentRef} className="relative flex-1 overflow-y-auto overflow-x-hidden min-h-0 bg-[#07080f]">
{images.length === 0 && loadingImages ? (
<div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8 absolute inset-0">
<div className="rounded-2xl border border-white/[0.06] bg-white/[0.02] p-8 min-w-72">
<div className="h-5 w-5 mx-auto rounded-full border-2 border-white/20 border-t-white/60 animate-spin" />
<p className="mt-4 text-sm text-white/40 font-medium">
{isSimilarResults
? "Finding similar images"
: searchMode === "semantic" && search.trim().length > 0
? `Searching for matches to "${search}"`
: parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
? `Searching for matches to "${parsedSearch.query}"`
: parsedSearch.mode === "tag" && parsedSearch.query.length > 0
? `Searching tags for "${parsedSearch.query}"`
: "Loading media"}
</p>
<p className="text-xs text-white/20 mt-1">
{isSimilarResults
? "Comparing visual embeddings"
: searchMode === "semantic" && search.trim().length > 0
: parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
? "Semantic search can take a little longer than filename search"
: parsedSearch.mode === "tag" && parsedSearch.query.length > 0
? "Matching against AI and user tags"
: "Fetching results"}
</p>
</div>
</div>
);
}
if (images.length === 0 && !loadingImages) {
return (
<div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8">
) : images.length === 0 && !loadingImages ? (
<div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8 absolute inset-0">
<div className="rounded-2xl border border-white/[0.06] bg-white/[0.02] p-8">
<svg className="h-12 w-12 mx-auto text-white/10 mb-4" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={0.75}
@@ -332,8 +345,10 @@ export function Gallery() {
? "Could not load results"
: isSimilarResults
? "No similar images found"
: searchMode === "semantic" && search.trim().length > 0
: parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
? "No semantic matches found"
: parsedSearch.mode === "tag" && parsedSearch.query.length > 0
? "No tag matches found"
: "No media found"}
</p>
<p className="text-xs text-white/15 mt-1">
@@ -341,17 +356,15 @@ export function Gallery() {
? imageLoadError
: isSimilarResults
? "This item may be visually isolated, or more embeddings may need to finish processing"
: searchMode === "semantic" && search.trim().length > 0
: parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
? "Try a broader phrase, or wait for more embeddings to finish processing"
: parsedSearch.mode === "tag" && parsedSearch.query.length > 0
? "Try a shorter tag, or wait for more tagging jobs to finish"
: "Try adjusting your filters or add a new folder"}
</p>
</div>
</div>
);
}
return (
<div ref={parentRef} className="relative flex-1 overflow-y-auto overflow-x-hidden min-h-0 bg-[#07080f]">
) : (
<div
className="grid content-start"
style={{
@@ -372,8 +385,9 @@ export function Gallery() {
/>
))}
</div>
)}
{loadingImages ? (
{images.length > 0 && loadingImages ? (
<div className="flex justify-center py-8">
<div className="h-4 w-4 rounded-full border-2 border-white/20 border-t-white/60 animate-spin" />
</div>
+243 -9
View File
@@ -58,28 +58,96 @@ function ratingPill(rating: AiRating): { label: string; className: string } {
}
}
interface DragRect {
startX: number;
startY: number;
endX: number;
endY: number;
}
/** Compute a CSS-pixel rect (relative to the viewport container) from a DragRect. */
function normaliseRect(r: DragRect): { left: number; top: number; width: number; height: number } {
return {
left: Math.min(r.startX, r.endX),
top: Math.min(r.startY, r.endY),
width: Math.abs(r.endX - r.startX),
height: Math.abs(r.endY - r.startY),
};
}
/** Convert a CSS-pixel drag rect (relative to viewport container) to normalised 01 crop coords
* relative to the actual rendered <img> element bounds. */
function rectToNormalisedCrop(
rect: DragRect,
imgEl: HTMLImageElement,
): { x: number; y: number; w: number; h: number } | null {
const imgBounds = imgEl.getBoundingClientRect();
if (imgBounds.width === 0 || imgBounds.height === 0) return null;
// rect coords are already in viewport space (client coords)
const rawX = Math.min(rect.startX, rect.endX);
const rawY = Math.min(rect.startY, rect.endY);
const rawW = Math.abs(rect.endX - rect.startX);
const rawH = Math.abs(rect.endY - rect.startY);
// Clamp to image bounds
const clampedX = Math.max(rawX, imgBounds.left);
const clampedY = Math.max(rawY, imgBounds.top);
const clampedRight = Math.min(rawX + rawW, imgBounds.right);
const clampedBottom = Math.min(rawY + rawH, imgBounds.bottom);
const croppedW = clampedRight - clampedX;
const croppedH = clampedBottom - clampedY;
if (croppedW <= 0 || croppedH <= 0) return null;
// Normalize by the CSS transform scale — getBoundingClientRect already returns
// the scaled (on-screen) size, so we normalize directly against that.
return {
x: (clampedX - imgBounds.left) / imgBounds.width,
y: (clampedY - imgBounds.top) / imgBounds.height,
w: croppedW / imgBounds.width,
h: croppedH / imgBounds.height,
};
}
/** Minimum selection size as a fraction of the viewport container dimension. */
const MIN_SELECTION_FRACTION = 0.02;
export function Lightbox() {
const selectedImage = useGalleryStore((state) => state.selectedImage);
const closeImage = useGalleryStore((state) => state.closeImage);
const images = useGalleryStore((state) => state.images);
const openImage = useGalleryStore((state) => state.openImage);
const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages);
const loadSimilarByRegion = useGalleryStore((state) => state.loadSimilarByRegion);
const similarScope = useGalleryStore((state) => state.similarScope);
const updateImageDetails = useGalleryStore((state) => state.updateImageDetails);
const getImageTags = useGalleryStore((state) => state.getImageTags);
const addUserTag = useGalleryStore((state) => state.addUserTag);
const removeTag = useGalleryStore((state) => state.removeTag);
const taggerModelStatus = useGalleryStore((state) => state.taggerModelStatus);
const queueTaggingForImage = useGalleryStore((state) => state.queueTaggingForImage);
const [zoom, setZoom] = useState(1);
const [imageTags, setImageTags] = useState<ImageTag[]>([]);
const [tagInput, setTagInput] = useState("");
const [tagAdding, setTagAdding] = useState(false);
const [tagsExpanded, setTagsExpanded] = useState(false);
const [taggingQueued, setTaggingQueued] = useState(false);
// Region selection state
const [regionSelectMode, setRegionSelectMode] = useState(false);
const [isDragging, setIsDragging] = useState(false);
const [dragRect, setDragRect] = useState<DragRect | null>(null);
const [regionSearching, setRegionSearching] = useState(false);
const imageViewportRef = useRef<HTMLDivElement>(null);
const imgRef = useRef<HTMLImageElement>(null);
const currentIndex = selectedImage ? images.findIndex((image) => image.id === selectedImage.id) : -1;
const canFindSimilar = selectedImage?.embedding_status === "ready";
const canSearchRegion = canFindSimilar && selectedImage?.media_kind === "image";
const goPrev = useCallback(() => {
if (currentIndex > 0) openImage(images[currentIndex - 1]);
@@ -89,13 +157,21 @@ export function Lightbox() {
if (currentIndex >= 0 && currentIndex < images.length - 1) openImage(images[currentIndex + 1]);
}, [currentIndex, images, openImage]);
const exitRegionMode = useCallback(() => {
setRegionSelectMode(false);
setIsDragging(false);
setDragRect(null);
}, []);
useEffect(() => {
setZoom(1);
setImageTags([]);
setTagInput("");
setTagsExpanded(false);
setTaggingQueued(false);
}, [selectedImage?.id]);
exitRegionMode();
setRegionSearching(false);
}, [selectedImage?.id, exitRegionMode]);
useEffect(() => {
if (!selectedImage) return;
@@ -114,6 +190,7 @@ export function Lightbox() {
if (!viewport || !selectedImage || selectedImage.media_kind !== "image") return;
const handleWheel = (event: WheelEvent) => {
if (regionSelectMode) return; // don't zoom during selection
if (!event.ctrlKey && Math.abs(event.deltaY) < Math.abs(event.deltaX)) return;
event.preventDefault();
setZoom((value) => {
@@ -124,12 +201,19 @@ export function Lightbox() {
viewport.addEventListener("wheel", handleWheel, { passive: false });
return () => viewport.removeEventListener("wheel", handleWheel);
}, [selectedImage]);
}, [selectedImage, regionSelectMode]);
useEffect(() => {
const handler = (event: KeyboardEvent) => {
if (!selectedImage) return;
if (event.key === "Escape") closeImage();
if (event.key === "Escape") {
if (regionSelectMode) {
exitRegionMode();
} else {
closeImage();
}
}
if (regionSelectMode) return; // block nav keys during selection
if (event.key === "ArrowLeft") goPrev();
if (event.key === "ArrowRight") goNext();
if (event.key === "+" || event.key === "=") setZoom((value) => Math.min(3, value + 0.25));
@@ -138,7 +222,75 @@ export function Lightbox() {
window.addEventListener("keydown", handler);
return () => window.removeEventListener("keydown", handler);
}, [selectedImage, closeImage, goPrev, goNext]);
}, [selectedImage, closeImage, goPrev, goNext, regionSelectMode, exitRegionMode]);
// ── Region selection pointer handlers ───────────────────────────────────────
const handleRegionPointerDown = useCallback(
(event: React.PointerEvent<HTMLDivElement>) => {
if (!regionSelectMode) return;
event.preventDefault();
event.currentTarget.setPointerCapture(event.pointerId);
setIsDragging(true);
setDragRect({
startX: event.clientX,
startY: event.clientY,
endX: event.clientX,
endY: event.clientY,
});
},
[regionSelectMode],
);
const handleRegionPointerMove = useCallback(
(event: React.PointerEvent<HTMLDivElement>) => {
if (!isDragging) return;
setDragRect((prev) =>
prev ? { ...prev, endX: event.clientX, endY: event.clientY } : null,
);
},
[isDragging],
);
const handleRegionPointerUp = useCallback(
(event: React.PointerEvent<HTMLDivElement>) => {
if (!isDragging || !dragRect || !selectedImage || !imgRef.current) {
setIsDragging(false);
return;
}
event.currentTarget.releasePointerCapture(event.pointerId);
const finalRect: DragRect = { ...dragRect, endX: event.clientX, endY: event.clientY };
const crop = rectToNormalisedCrop(finalRect, imgRef.current);
setIsDragging(false);
setDragRect(null);
// Ignore tiny accidental clicks
const containerBounds = imageViewportRef.current?.getBoundingClientRect();
const containerSize = containerBounds
? Math.min(containerBounds.width, containerBounds.height)
: 500;
const selW = Math.abs(finalRect.endX - finalRect.startX);
const selH = Math.abs(finalRect.endY - finalRect.startY);
if (!crop || selW < containerSize * MIN_SELECTION_FRACTION || selH < containerSize * MIN_SELECTION_FRACTION) {
exitRegionMode();
return;
}
exitRegionMode();
setRegionSearching(true);
const folderId =
similarScope === "current_folder" ? selectedImage.folder_id : null;
void loadSimilarByRegion(selectedImage.id, crop, folderId, selectedImage.folder_id)
.finally(() => setRegionSearching(false));
},
[isDragging, dragRect, selectedImage, similarScope, loadSimilarByRegion, exitRegionMode],
);
// Build the CSS rect for the selection overlay (viewport-relative)
const selectionOverlay =
isDragging && dragRect ? normaliseRect(dragRect) : null;
return (
<AnimatePresence>
@@ -150,11 +302,11 @@ export function Lightbox() {
animate={{ opacity: 1 }}
exit={{ opacity: 0 }}
transition={{ duration: 0.15 }}
onClick={closeImage}
onClick={regionSelectMode ? undefined : closeImage}
>
<button
className="absolute left-4 top-1/2 z-10 -translate-y-1/2 rounded-full bg-white/10 p-3 text-white transition-colors hover:bg-white/20 disabled:opacity-20"
disabled={currentIndex <= 0}
disabled={currentIndex <= 0 || regionSelectMode}
onClick={(event) => {
event.stopPropagation();
goPrev();
@@ -169,8 +321,38 @@ export function Lightbox() {
<div className="flex flex-1 overflow-hidden">
<div
ref={imageViewportRef}
className="group relative flex flex-1 items-center justify-center overflow-auto p-10"
className={`group relative flex flex-1 items-center justify-center overflow-auto p-10 ${
regionSelectMode ? "cursor-crosshair select-none" : ""
}`}
onPointerDown={handleRegionPointerDown}
onPointerMove={handleRegionPointerMove}
onPointerUp={handleRegionPointerUp}
>
{/* Region selection mode hint */}
{regionSelectMode && (
<div className="pointer-events-none absolute inset-x-0 top-4 z-20 flex justify-center">
<div className="flex items-center gap-2 rounded-full border border-white/15 bg-black/70 px-4 py-2 text-xs text-gray-300 backdrop-blur">
<svg className="h-3.5 w-3.5 text-violet-400" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 8V4m0 0h4M4 4l5 5m11-1V4m0 0h-4m4 0l-5 5M4 16v4m0 0h4m-4 0l5-5m11 5l-5-5m5 5v-4m0 4h-4" />
</svg>
Draw a region to search <kbd className="ml-1 rounded border border-white/20 bg-white/10 px-1.5 py-0.5 font-mono text-[10px]">Esc</kbd> to cancel
</div>
</div>
)}
{/* Selection rectangle overlay */}
{selectionOverlay && selectionOverlay.width > 4 && selectionOverlay.height > 4 && (
<div
className="pointer-events-none fixed z-30 rounded border-2 border-violet-400 bg-violet-400/15 shadow-[0_0_0_9999px_rgba(0,0,0,0.35)]"
style={{
left: selectionOverlay.left,
top: selectionOverlay.top,
width: selectionOverlay.width,
height: selectionOverlay.height,
}}
/>
)}
<AnimatePresence mode="wait">
<motion.div
key={selectedImage.id}
@@ -190,6 +372,7 @@ export function Lightbox() {
) : (
<>
<img
ref={imgRef}
src={convertFileSrc(selectedImage.path)}
alt={selectedImage.filename}
className="max-w-full rounded-2xl shadow-2xl"
@@ -197,8 +380,11 @@ export function Lightbox() {
maxHeight: "calc(100vh - 10rem)",
transform: `scale(${zoom})`,
transformOrigin: "center center",
// Slightly dim the image while in region select mode
...(regionSelectMode ? { opacity: 0.85 } : {}),
}}
/>
{!regionSelectMode && (
<div className="pointer-events-none absolute right-6 top-6 opacity-0 transition-opacity group-hover:opacity-100">
<div className="pointer-events-auto flex items-center gap-1 rounded-full border border-white/10 bg-black/55 px-2 py-1 backdrop-blur">
<button
@@ -216,6 +402,7 @@ export function Lightbox() {
</button>
</div>
</div>
)}
</>
)}
</motion.div>
@@ -246,7 +433,7 @@ export function Lightbox() {
}`}
onClick={() => {
if (!canFindSimilar) return;
void loadSimilarImages(selectedImage.id);
void loadSimilarImages(selectedImage.id, similarScope === "current_folder" ? selectedImage.folder_id : null, true, selectedImage.folder_id);
}}
disabled={!canFindSimilar}
>
@@ -260,6 +447,53 @@ export function Lightbox() {
</button>
</div>
{/* Search region button row */}
{canSearchRegion && (
<div className="shrink-0 px-5 pb-3">
<button
className={`w-full rounded-lg border px-3 py-2 text-xs transition-colors ${
regionSelectMode
? "border-violet-400/40 bg-violet-500/15 text-violet-300 hover:bg-violet-500/20"
: regionSearching
? "border-white/5 bg-white/[0.03] text-gray-500 cursor-not-allowed"
: "border-white/10 bg-white/5 text-gray-300 hover:bg-white/10 hover:text-white"
}`}
onClick={() => {
if (regionSearching) return;
setRegionSelectMode((prev) => !prev);
setDragRect(null);
setIsDragging(false);
}}
disabled={regionSearching}
title={regionSelectMode ? "Cancel region selection" : "Draw a region on the image to search for similar content"}
>
{regionSearching ? (
<span className="flex items-center justify-center gap-1.5">
<svg className="h-3 w-3 animate-spin" fill="none" viewBox="0 0 24 24">
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4" />
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8v8H4z" />
</svg>
Searching region
</span>
) : regionSelectMode ? (
<span className="flex items-center justify-center gap-1.5">
<svg className="h-3 w-3" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</svg>
Cancel selection
</span>
) : (
<span className="flex items-center justify-center gap-1.5">
<svg className="h-3 w-3" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 8V4m0 0h4M4 4l5 5m11-1V4m0 0h-4m4 0l-5 5M4 16v4m0 0h4m-4 0l5-5m11 5l-5-5m5 5v-4m0 4h-4" />
</svg>
Search within image
</span>
)}
</button>
</div>
)}
<div className="min-h-0 flex-1 overflow-y-auto px-5 pb-4 space-y-4 text-sm">
<div>
<p className="mb-1 text-xs uppercase tracking-wider text-gray-500">Rating</p>
@@ -471,7 +705,7 @@ export function Lightbox() {
<button
className="absolute right-80 top-1/2 z-10 -translate-y-1/2 rounded-full bg-white/10 p-3 text-white transition-colors hover:bg-white/20 disabled:opacity-20"
disabled={currentIndex >= images.length - 1}
disabled={currentIndex >= images.length - 1 || regionSelectMode}
onClick={(event) => {
event.stopPropagation();
goNext();
+355 -249
View File
@@ -1,16 +1,13 @@
import { useEffect, useState } from "react";
import { TaggerAcceleration, useGalleryStore } from "../store";
import { useEffect, useMemo, useState } from "react";
import { TaggerAcceleration, TaggingQueueScope, useGalleryStore } from "../store";
type SettingsSection = "tagging" | "library" | "display" | "storage";
type SettingsSection = "workspace" | "workers";
const SECTIONS: { id: SettingsSection; label: string; detail: string }[] = [
{ id: "tagging", label: "AI Tagging", detail: "WD tagger model" },
{ id: "library", label: "Library", detail: "Indexing and scanning" },
{ id: "display", label: "Display", detail: "Gallery preferences" },
{ id: "storage", label: "Storage", detail: "Cache and model files" },
{ id: "workspace", label: "AI Workspace", detail: "Tagging models and queue targets" },
{ id: "workers", label: "Workers", detail: "Queue activity and background processing" },
];
function StatusPill({ children, tone }: { children: React.ReactNode; tone: "ready" | "muted" | "busy" }) {
const className =
tone === "ready"
@@ -19,19 +16,72 @@ function StatusPill({ children, tone }: { children: React.ReactNode; tone: "read
? "border-sky-400/25 bg-sky-500/10 text-sky-300"
: "border-white/10 bg-white/[0.04] text-gray-500";
return <span className={`inline-flex rounded-md border px-2 py-1 text-[11px] font-medium ${className}`}>{children}</span>;
}
function SectionShell({ eyebrow, title, description, children }: {
eyebrow: string;
title: string;
description?: string;
children: React.ReactNode;
}) {
return (
<span className={`inline-flex rounded-md border px-2 py-1 text-[11px] font-medium ${className}`}>
{children}
</span>
<section>
<p className="text-[10px] font-semibold uppercase tracking-[0.18em] text-gray-600">{eyebrow}</p>
<h3 className="mt-1 text-lg font-semibold text-white">{title}</h3>
{description ? <p className="mt-2 max-w-2xl text-sm leading-relaxed text-gray-500">{description}</p> : null}
<div className="mt-5 space-y-4">{children}</div>
</section>
);
}
function TaggerAccelerationButton({
acceleration,
current,
onSelect,
children,
}: {
function SettingsCard({ title, description, children }: { title: string; description?: string; children: React.ReactNode }) {
return (
<div className="rounded-2xl border border-white/[0.07] bg-white/[0.03] p-5">
<div className="flex flex-col gap-1 border-b border-white/[0.07] pb-4">
<p className="text-sm font-medium text-white">{title}</p>
{description ? <p className="text-xs leading-relaxed text-gray-500">{description}</p> : null}
</div>
<div className="pt-4">{children}</div>
</div>
);
}
function SettingsRow({ title, description, children }: { title: string; description: string; children: React.ReactNode }) {
return (
<div className="flex items-start justify-between gap-5 border-b border-white/[0.07] py-4 last:border-b-0 first:pt-0 last:pb-0">
<div className="min-w-0">
<p className="text-sm font-medium text-white">{title}</p>
<p className="mt-1 max-w-md text-xs leading-relaxed text-gray-500">{description}</p>
</div>
<div className="shrink-0">{children}</div>
</div>
);
}
function ScopeButton({ scope, current, onSelect, children }: {
scope: TaggingQueueScope;
current: TaggingQueueScope;
onSelect: (scope: TaggingQueueScope) => void;
children: React.ReactNode;
}) {
const active = scope === current;
return (
<button
type="button"
className={`rounded-md border px-3 py-1.5 text-xs transition-colors ${
active
? "border-emerald-400/35 bg-emerald-500/15 text-emerald-200"
: "border-white/10 bg-white/[0.045] text-gray-500 hover:bg-white/[0.075] hover:text-gray-200"
}`}
onClick={() => onSelect(scope)}
>
{children}
</button>
);
}
function TaggerAccelerationButton({ acceleration, current, onSelect, children }: {
acceleration: TaggerAcceleration;
current: TaggerAcceleration;
onSelect: (acceleration: TaggerAcceleration) => void;
@@ -53,63 +103,32 @@ function TaggerAccelerationButton({
);
}
function SettingsRow({
title,
description,
children,
}: {
title: string;
description: string;
children: React.ReactNode;
}) {
return (
<div className="flex items-start justify-between gap-5 border-b border-white/[0.07] py-4 last:border-b-0">
<div className="min-w-0">
<p className="text-sm font-medium text-white">{title}</p>
<p className="mt-1 max-w-md text-xs leading-relaxed text-gray-500">{description}</p>
</div>
<div className="shrink-0">{children}</div>
</div>
);
}
function SectionShell({
eyebrow,
title,
children,
}: {
eyebrow: string;
title: string;
children: React.ReactNode;
}) {
return (
<div>
<p className="text-[10px] font-semibold uppercase tracking-[0.18em] text-gray-600">{eyebrow}</p>
<h3 className="mt-1 text-lg font-semibold text-white">{title}</h3>
<div className="mt-5 border-t border-white/[0.08]">{children}</div>
</div>
);
}
export function SettingsModal() {
const [activeSection, setActiveSection] = useState<SettingsSection>("tagging");
const [activeSection, setActiveSection] = useState<SettingsSection>("workspace");
const [taggerQueueStatus, setTaggerQueueStatus] = useState<string | null>(null);
const [taggerQueueing, setTaggerQueueing] = useState(false);
const [taggerClearing, setTaggerClearing] = useState(false);
const [taggerAccelerationSaving, setTaggerAccelerationSaving] = useState(false);
const [taggerThresholdDraft, setTaggerThresholdDraft] = useState<string | null>(null);
const [taggerThresholdSaving, setTaggerThresholdSaving] = useState(false);
const [taggerBatchSizeDraft, setTaggerBatchSizeDraft] = useState<string | null>(null);
const [taggerBatchSizeSaving, setTaggerBatchSizeSaving] = useState(false);
const settingsOpen = useGalleryStore((state) => state.settingsOpen);
const setSettingsOpen = useGalleryStore((state) => state.setSettingsOpen);
const selectedFolderId = useGalleryStore((state) => state.selectedFolderId);
const folders = useGalleryStore((state) => state.folders);
const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress);
const taggingQueueScope = useGalleryStore((state) => state.taggingQueueScope);
const taggingQueueFolderIds = useGalleryStore((state) => state.taggingQueueFolderIds);
const setTaggingQueueScope = useGalleryStore((state) => state.setTaggingQueueScope);
const toggleTaggingQueueFolder = useGalleryStore((state) => state.toggleTaggingQueueFolder);
const setTaggingQueueFolderIds = useGalleryStore((state) => state.setTaggingQueueFolderIds);
const taggerModelStatus = useGalleryStore((state) => state.taggerModelStatus);
const taggerModelPreparing = useGalleryStore((state) => state.taggerModelPreparing);
const taggerModelProgress = useGalleryStore((state) => state.taggerModelProgress);
const taggerModelError = useGalleryStore((state) => state.taggerModelError);
const taggerAcceleration = useGalleryStore((state) => state.taggerAcceleration);
const taggerThreshold = useGalleryStore((state) => state.taggerThreshold);
const taggerBatchSize = useGalleryStore((state) => state.taggerBatchSize);
const taggerRuntimeProbe = useGalleryStore((state) => state.taggerRuntimeProbe);
const taggerRuntimeChecking = useGalleryStore((state) => state.taggerRuntimeChecking);
const loadTaggerModelStatus = useGalleryStore((state) => state.loadTaggerModelStatus);
@@ -119,49 +138,123 @@ export function SettingsModal() {
const setTaggerAcceleration = useGalleryStore((state) => state.setTaggerAcceleration);
const loadTaggerThreshold = useGalleryStore((state) => state.loadTaggerThreshold);
const setTaggerThreshold = useGalleryStore((state) => state.setTaggerThreshold);
const loadTaggerBatchSize = useGalleryStore((state) => state.loadTaggerBatchSize);
const setTaggerBatchSize = useGalleryStore((state) => state.setTaggerBatchSize);
const probeTaggerRuntime = useGalleryStore((state) => state.probeTaggerRuntime);
const queueTaggingJobs = useGalleryStore((state) => state.queueTaggingJobs);
const queueTaggingJobsForFolders = useGalleryStore((state) => state.queueTaggingJobsForFolders);
const clearTaggingJobs = useGalleryStore((state) => state.clearTaggingJobs);
const clearTaggingJobsForFolders = useGalleryStore((state) => state.clearTaggingJobsForFolders);
useEffect(() => {
if (!settingsOpen) return;
void loadTaggerModelStatus();
void loadTaggerAcceleration();
void loadTaggerThreshold();
void loadTaggerBatchSize();
const handleKeyDown = (event: KeyboardEvent) => {
if (event.key === "Escape") setSettingsOpen(false);
};
window.addEventListener("keydown", handleKeyDown);
return () => window.removeEventListener("keydown", handleKeyDown);
}, [settingsOpen, loadTaggerModelStatus, loadTaggerAcceleration, loadTaggerThreshold, setSettingsOpen]);
}, [settingsOpen, loadTaggerModelStatus, loadTaggerAcceleration, loadTaggerThreshold, loadTaggerBatchSize, setSettingsOpen]);
const selectedFolders = useMemo(
() => folders.filter((folder) => taggingQueueFolderIds.includes(folder.id)),
[folders, taggingQueueFolderIds],
);
const totalQueuedJobs = useMemo(
() => Object.values(mediaJobProgress).reduce((sum, progress) => sum + (progress?.tagging_pending ?? 0), 0),
[mediaJobProgress],
);
if (!settingsOpen) return null;
const selectedFolder = folders.find((folder) => folder.id === selectedFolderId);
const scopeLabel = selectedFolder ? selectedFolder.name : "all libraries";
const taggerReady = taggerModelStatus?.ready ?? false;
const queueScopeLabel =
taggingQueueScope === "all"
? "all media"
: selectedFolders.length > 0
? `${selectedFolders.length} selected folder${selectedFolders.length === 1 ? "" : "s"}`
: "no folders selected";
const thresholdDisplay = taggerThresholdDraft ?? String(taggerThreshold);
const batchSizeDisplay = taggerBatchSizeDraft ?? String(taggerBatchSize);
const taggerDownloadLabel = taggerModelProgress
? `Downloading ${taggerModelProgress.completed_files}/${taggerModelProgress.total_files}`
: taggerModelPreparing
? "Preparing WD Tagger..."
: taggerReady
? "Installed"
: "Install model";
const taggerDownloadPercent = taggerModelProgress
? Math.round((taggerModelProgress.completed_files / Math.max(taggerModelProgress.total_files, 1)) * 100)
: 0;
const runQueueAction = (action: "queue" | "clear") => {
const selectedIds = taggingQueueFolderIds;
const perform =
taggingQueueScope === "all"
? action === "queue"
? queueTaggingJobs(null)
: clearTaggingJobs(null)
: selectedIds.length > 0
? action === "queue"
? queueTaggingJobsForFolders(selectedIds)
: clearTaggingJobsForFolders(selectedIds)
: Promise.resolve(0);
if (action === "queue") {
setTaggerQueueing(true);
} else {
setTaggerClearing(true);
}
setTaggerQueueStatus(null);
void perform
.then((count) => {
if (taggingQueueScope === "selected" && selectedIds.length === 0) {
setTaggerQueueStatus("Choose at least one folder before running tagging jobs.");
return;
}
setTaggerQueueStatus(
count === 0
? action === "queue"
? "No missing tags found for the current target."
: "No queued tagging jobs to clear for the current target."
: action === "queue"
? `Queued ${count.toLocaleString()} image${count === 1 ? "" : "s"} for tagging.`
: `Cleared ${count.toLocaleString()} queued tagging job${count === 1 ? "" : "s"}.`,
);
})
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => {
if (action === "queue") {
setTaggerQueueing(false);
} else {
setTaggerClearing(false);
}
});
};
return (
<div className="fixed inset-0 z-[60] flex items-center justify-center bg-black/65 px-6 backdrop-blur-sm" onClick={() => setSettingsOpen(false)}>
<div
className="fixed inset-0 z-[60] flex items-center justify-center bg-black/65 px-6 backdrop-blur-sm"
onClick={() => setSettingsOpen(false)}
>
<div
className="flex h-[min(680px,calc(100vh-56px))] w-full max-w-4xl overflow-hidden rounded-lg border border-white/10 bg-[#07080f] shadow-2xl shadow-black/60"
className="flex h-[min(760px,calc(100vh-56px))] w-full max-w-5xl overflow-hidden rounded-lg border border-white/10 bg-[#07080f] shadow-2xl shadow-black/60"
onClick={(event) => event.stopPropagation()}
>
<aside className="flex w-56 shrink-0 flex-col border-r border-white/[0.07] bg-white/[0.025]">
<aside className="flex w-64 shrink-0 flex-col border-r border-white/[0.07] bg-white/[0.025]">
<div className="border-b border-white/[0.07] px-5 py-5">
<p className="text-base font-semibold text-white">Settings</p>
<p className="mt-1 text-xs text-gray-600">Phokus preferences</p>
<p className="mt-1 text-xs text-gray-600">Operational controls for AI workflows</p>
</div>
<div className="flex-1 space-y-1 overflow-y-auto p-2">
{SECTIONS.map((section) => (
<button
key={section.id}
className={`w-full rounded-md px-3 py-2.5 text-left transition-colors ${
activeSection === section.id
? "bg-white/10 text-white"
: "text-gray-500 hover:bg-white/[0.055] hover:text-gray-200"
activeSection === section.id ? "bg-white/10 text-white" : "text-gray-500 hover:bg-white/[0.055] hover:text-gray-200"
}`}
onClick={() => setActiveSection(section.id)}
>
@@ -173,7 +266,11 @@ export function SettingsModal() {
</aside>
<main className="flex min-w-0 flex-1 flex-col">
<div className="flex h-14 shrink-0 items-center justify-end border-b border-white/[0.07] px-6">
<div className="flex h-14 shrink-0 items-center justify-between border-b border-white/[0.07] px-6">
<div>
<p className="text-sm font-medium text-white">{activeSection === "workspace" ? "AI Workspace" : "Workers"}</p>
<p className="text-xs text-gray-600">{activeSection === "workspace" ? "Model setup and queue targets" : "Background processing status"}</p>
</div>
<button
className="rounded-md p-1.5 text-gray-500 transition-colors hover:bg-white/[0.06] hover:text-white"
onClick={() => setSettingsOpen(false)}
@@ -186,37 +283,33 @@ export function SettingsModal() {
</div>
<div className="flex-1 overflow-y-auto px-7 py-6">
{activeSection === "tagging" ? (() => {
const taggerReady = taggerModelStatus?.ready ?? false;
const taggerDownloadLabel = taggerModelProgress
? `Downloading ${taggerModelProgress.completed_files}/${taggerModelProgress.total_files}`
: taggerModelPreparing
? "Preparing WD Tagger..."
: taggerReady
? "Downloaded"
: "Download WD Tagger";
const taggerDownloadPercent = taggerModelProgress
? Math.round((taggerModelProgress.completed_files / Math.max(taggerModelProgress.total_files, 1)) * 100)
: 0;
const thresholdDisplay = taggerThresholdDraft ?? String(taggerThreshold);
return (
<SectionShell eyebrow="AI Tagging" title="WD SwinV2 Tagger v3">
<SettingsRow
title="WD Tagger model"
description={taggerReady ? "Stored locally and available offline." : "Download the model to enable automatic AI tagging."}
{activeSection === "workspace" ? (
<div className="space-y-8">
<SectionShell
eyebrow="AI Workspace"
title="Tagging"
>
<SettingsCard title="Tagging Models">
<div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
<div className="flex items-start justify-between gap-4">
<div>
<div className="flex items-center gap-2">
<p className="text-sm font-medium text-white">WD SwinV2 Tagger v3</p>
<StatusPill tone={taggerReady ? "ready" : taggerModelPreparing ? "busy" : "muted"}>
{taggerReady ? "Installed" : taggerModelPreparing ? "Preparing" : "Not installed"}
</StatusPill>
</div>
<p className="mt-1 max-w-xl text-xs leading-relaxed text-gray-500">
Anime-focused vision model by SmilingWolf. Generates booru-style tags with configurable confidence thresholds.
</p>
</div>
<div className="flex flex-col items-end gap-2">
<button
className="relative overflow-hidden rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void prepareTaggerModel()}
disabled={taggerModelPreparing || taggerReady}
>
{taggerModelProgress ? (
<span
className="absolute inset-y-0 left-0 bg-emerald-400/15 transition-[width] duration-200"
style={{ width: `${taggerDownloadPercent}%` }}
/>
) : null}
{taggerModelProgress ? <span className="absolute inset-y-0 left-0 bg-emerald-400/15 transition-[width] duration-200" style={{ width: `${taggerDownloadPercent}%` }} /> : null}
<span className="relative">{taggerDownloadLabel}</span>
</button>
{taggerReady ? (
@@ -238,63 +331,34 @@ export function SettingsModal() {
</>
) : null}
</div>
</SettingsRow>
</div>
<SettingsRow
title="Tagger acceleration"
description="Use DirectML for GPU-accelerated tagging when available."
>
<div className="mt-4 space-y-4 border-t border-white/[0.07] pt-4">
<SettingsRow title="Tagger acceleration" description="Use DirectML when available, or fall back to CPU for reliability.">
<div className="flex flex-col items-end gap-2">
<div className="flex rounded-lg border border-white/[0.07] bg-black/20 p-1">
{(["auto", "directml", "cpu"] as const).map((acceleration) => (
<TaggerAccelerationButton
acceleration="auto"
key={acceleration}
acceleration={acceleration}
current={taggerAcceleration}
onSelect={(acceleration) => {
onSelect={(nextAcceleration) => {
setTaggerAccelerationSaving(true);
void setTaggerAcceleration(acceleration)
void setTaggerAcceleration(nextAcceleration)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerAccelerationSaving(false));
}}
>
Auto
</TaggerAccelerationButton>
<TaggerAccelerationButton
acceleration="directml"
current={taggerAcceleration}
onSelect={(acceleration) => {
setTaggerAccelerationSaving(true);
void setTaggerAcceleration(acceleration)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerAccelerationSaving(false));
}}
>
DirectML
</TaggerAccelerationButton>
<TaggerAccelerationButton
acceleration="cpu"
current={taggerAcceleration}
onSelect={(acceleration) => {
setTaggerAccelerationSaving(true);
void setTaggerAcceleration(acceleration)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerAccelerationSaving(false));
}}
>
CPU
{acceleration === "directml" ? "DirectML" : acceleration === "cpu" ? "CPU" : "Auto"}
</TaggerAccelerationButton>
))}
</div>
<p className="text-[11px] text-gray-600">
{taggerAccelerationSaving ? "Saving..." : `Current: ${taggerAcceleration}`}
</p>
<p className="text-[11px] text-gray-600">{taggerAccelerationSaving ? "Saving..." : `Current: ${taggerAcceleration}`}</p>
</div>
</SettingsRow>
<SettingsRow
title="Confidence threshold"
description="Tags with confidence below this value are discarded. Lower = more tags, higher = fewer but more accurate."
>
<SettingsRow title="Confidence threshold" description="Lower values keep more tags. Higher values are stricter and usually cleaner.">
<div className="flex flex-col items-end gap-2">
<div className="flex items-center gap-2">
<input
type="number"
min="0.05"
@@ -302,12 +366,12 @@ export function SettingsModal() {
step="0.05"
className="w-20 rounded-md border border-white/10 bg-white/5 px-2 py-1.5 text-xs text-white focus:border-white/20 focus:outline-none"
value={thresholdDisplay}
onChange={(e) => setTaggerThresholdDraft(e.target.value)}
onChange={(event) => setTaggerThresholdDraft(event.target.value)}
onBlur={() => {
const val = parseFloat(thresholdDisplay);
if (!isNaN(val) && val >= 0.05 && val <= 0.99) {
const value = parseFloat(thresholdDisplay);
if (!isNaN(value) && value >= 0.05 && value <= 0.99) {
setTaggerThresholdSaving(true);
void setTaggerThreshold(val)
void setTaggerThreshold(value)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => {
setTaggerThresholdDraft(null);
@@ -318,142 +382,184 @@ export function SettingsModal() {
}
}}
/>
</div>
<p className="text-[11px] text-gray-600">
{taggerThresholdSaving ? "Saving..." : "Default: 0.35"}
</p>
<p className="text-[11px] text-gray-600">{taggerThresholdSaving ? "Saving..." : "Default: 0.35"}</p>
</div>
</SettingsRow>
<SettingsRow
title="Tagging queue"
description={`Generate missing AI tags in ${scopeLabel}. Tags update as the background worker finishes each image.`}
>
<SettingsRow title="Tagging batch size" description="Number of images processed concurrently during tag generation.">
<div className="flex flex-col items-end gap-2">
<button
className="rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => {
setTaggerQueueing(true);
setTaggerQueueStatus(null);
void queueTaggingJobs(selectedFolderId)
.then((queued) => {
setTaggerQueueStatus(
queued === 0
? "No missing tags found."
: `Queued ${queued.toLocaleString()} image${queued === 1 ? "" : "s"}.`,
);
})
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerQueueing(false));
<input
type="number"
min="1"
max="100"
step="1"
className="w-20 rounded-md border border-white/10 bg-white/5 px-2 py-1.5 text-xs text-white focus:border-white/20 focus:outline-none"
value={batchSizeDisplay}
onChange={(event) => setTaggerBatchSizeDraft(event.target.value)}
onBlur={() => {
const value = parseInt(batchSizeDisplay, 10);
if (!isNaN(value) && value >= 1 && value <= 100) {
setTaggerBatchSizeSaving(true);
void setTaggerBatchSize(value)
.catch((error: unknown) => setTaggerQueueStatus(String(error)))
.finally(() => {
setTaggerBatchSizeDraft(null);
setTaggerBatchSizeSaving(false);
});
} else {
setTaggerBatchSizeDraft(null);
}
}}
disabled={!taggerReady || taggerQueueing || taggerClearing}
>
{taggerQueueing
? "Queueing..."
: selectedFolder
? "Tag this library"
: "Tag all libraries"}
</button>
<button
className="rounded-md border border-amber-400/20 bg-amber-500/10 px-3 py-1.5 text-xs text-amber-200 transition-colors hover:bg-amber-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => {
setTaggerClearing(true);
setTaggerQueueStatus(null);
void clearTaggingJobs(selectedFolderId)
.then((cleared) => {
setTaggerQueueStatus(
cleared === 0
? "No queued tagging jobs to clear."
: `Cleared ${cleared.toLocaleString()} queued job${cleared === 1 ? "" : "s"}.`,
);
})
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerClearing(false));
}}
disabled={taggerQueueing || taggerClearing}
>
{taggerClearing
? "Clearing..."
: selectedFolder
? "Clear this queue"
: "Clear all queued tags"}
</button>
/>
<p className="text-[11px] text-gray-600">{taggerBatchSizeSaving ? "Saving..." : "Default: 8"}</p>
</div>
</SettingsRow>
<div className="py-4">
<div>
<p className="text-xs font-medium text-gray-400">Model location</p>
<p className="mt-2 break-all rounded-md border border-white/[0.07] bg-black/20 px-3 py-2 text-xs text-gray-600">
{taggerReady ? taggerModelStatus?.local_dir : "Not downloaded"}
</p>
{taggerModelProgress?.current_file ? (
<p className="mt-3 break-all text-xs text-gray-500">{taggerModelProgress.current_file}</p>
) : null}
{taggerModelError ? (
<p className="mt-3 text-xs text-amber-300">{taggerModelError}</p>
) : null}
{taggerQueueStatus ? (
<p className="mt-3 text-xs text-gray-500">{taggerQueueStatus}</p>
) : null}
{taggerModelProgress?.current_file ? <p className="mt-3 break-all text-xs text-gray-500">{taggerModelProgress.current_file}</p> : null}
{taggerModelError ? <p className="mt-3 text-xs text-amber-300">{taggerModelError}</p> : null}
{taggerRuntimeProbe ? (
<div className="mt-4 border-t border-white/[0.07] pt-4">
<div className="mt-4 rounded-lg border border-white/[0.07] bg-black/20 px-3 py-3">
<div className="flex items-center justify-between gap-3">
<p className="text-xs font-medium text-gray-400">Runtime check</p>
<StatusPill tone="ready">Ready</StatusPill>
</div>
<p className="mt-2 text-xs text-gray-600">
Tagger acceleration: {taggerRuntimeProbe.acceleration}
</p>
<div className="mt-3 space-y-2">
<div className="rounded-md border border-white/[0.07] bg-black/20 px-3 py-2">
<p className="break-all text-xs text-gray-400">{taggerRuntimeProbe.session.file}</p>
<p className="mt-1 text-[11px] text-gray-600">
{taggerRuntimeProbe.session.inputs.length} input{taggerRuntimeProbe.session.inputs.length === 1 ? "" : "s"} · {taggerRuntimeProbe.session.outputs.join(", ")}
</p>
</div>
</div>
<p className="mt-2 text-xs text-gray-600">Tagger acceleration: {taggerRuntimeProbe.acceleration}</p>
<p className="mt-2 break-all text-xs text-gray-500">{taggerRuntimeProbe.session.file}</p>
</div>
) : null}
</div>
</SectionShell>
);
})() : null}
</div>
</div>
</SettingsCard>
{activeSection === "library" ? (
<SectionShell eyebrow="Library" title="Indexing and scanning">
<SettingsRow title="Background workers" description="Folder-level pause controls remain in the background tasks panel.">
<StatusPill tone="muted">Managed per folder</StatusPill>
<SettingsCard title="Queue Targets" description="Choose which folders to include when queuing tagging jobs.">
<SettingsRow title="Target scope" description="Queue across the full library, or choose a specific folder set and keep the modal open while you work through it.">
<div className="flex rounded-lg border border-white/[0.07] bg-black/20 p-1">
<ScopeButton scope="all" current={taggingQueueScope} onSelect={setTaggingQueueScope}>All media</ScopeButton>
<ScopeButton scope="selected" current={taggingQueueScope} onSelect={setTaggingQueueScope}>Selected folders</ScopeButton>
</div>
</SettingsRow>
<SettingsRow title="Reindexing" description="Use the library sidebar to rescan a folder when files change.">
<StatusPill tone="muted">Available</StatusPill>
</SettingsRow>
</SectionShell>
) : null}
{activeSection === "display" ? (
<SectionShell eyebrow="Display" title="Gallery preferences">
<SettingsRow title="Grid density" description="Use the toolbar size control to change thumbnail density.">
<StatusPill tone="muted">Toolbar</StatusPill>
</SettingsRow>
<SettingsRow title="Result view" description="Similar results reset to the top on each new search.">
<StatusPill tone="ready">Enabled</StatusPill>
</SettingsRow>
</SectionShell>
) : null}
{activeSection === "storage" ? (
<SectionShell eyebrow="Storage" title="Local files">
<SettingsRow title="WD Tagger" description="Remove the local tagger model without changing the rest of the library.">
<div className="border-b border-white/[0.07] py-4">
<div className="flex items-center justify-between gap-4">
<div>
<p className="text-sm font-medium text-white">Folder selection</p>
<p className="mt-1 text-xs leading-relaxed text-gray-500">Current target: {queueScopeLabel}.</p>
</div>
<div className="flex gap-2">
<button
className="rounded-md border border-red-400/20 bg-red-500/10 px-3 py-1.5 text-xs text-red-200 transition-colors hover:bg-red-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void deleteTaggerModel()}
disabled={taggerModelPreparing || !(taggerModelStatus?.ready ?? false)}
className="rounded-md border border-white/10 bg-white/[0.045] px-2.5 py-1 text-[11px] text-gray-400 transition-colors hover:bg-white/[0.075] hover:text-white disabled:opacity-40"
onClick={() => setTaggingQueueFolderIds(folders.map((folder) => folder.id))}
disabled={folders.length === 0}
>
Delete model files
Select all
</button>
<button
className="rounded-md border border-white/10 bg-white/[0.045] px-2.5 py-1 text-[11px] text-gray-400 transition-colors hover:bg-white/[0.075] hover:text-white disabled:opacity-40"
onClick={() => setTaggingQueueFolderIds([])}
disabled={taggingQueueFolderIds.length === 0}
>
Clear
</button>
</div>
</div>
<div className={`mt-4 grid max-h-64 gap-2 overflow-y-auto pr-1 ${taggingQueueScope === "selected" ? "opacity-100" : "opacity-60"}`}>
{folders.map((folder) => {
const active = taggingQueueFolderIds.includes(folder.id);
const progress = mediaJobProgress[folder.id];
return (
<button
key={folder.id}
type="button"
className={`flex items-center justify-between rounded-xl border px-3 py-2 text-left transition-colors ${
active
? "border-emerald-400/30 bg-emerald-500/10 text-white"
: "border-white/[0.07] bg-black/20 text-gray-300 hover:border-white/15 hover:bg-white/[0.04]"
}`}
onClick={() => toggleTaggingQueueFolder(folder.id)}
>
<div>
<p className="text-sm font-medium">{folder.name}</p>
<p className="mt-1 text-[11px] text-gray-500">{folder.image_count.toLocaleString()} items</p>
</div>
<div className="flex items-center gap-2">
{(progress?.tagging_pending ?? 0) > 0 ? <StatusPill tone="busy">{progress?.tagging_pending} queued</StatusPill> : null}
<span className={`h-4 w-4 rounded-full border ${active ? "border-emerald-300 bg-emerald-300" : "border-white/15 bg-transparent"}`} />
</div>
</button>
);
})}
{folders.length === 0 ? <p className="text-sm text-gray-500">Add a folder first to enable targeted tagging queues.</p> : null}
</div>
</div>
<SettingsRow title="Queue tagging jobs" description="Generate missing AI tags for the current target. Results flow back into the library as the background worker finishes.">
<div className="flex flex-col items-end gap-2">
<button
className="rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => runQueueAction("queue")}
disabled={!taggerReady || taggerQueueing || taggerClearing || (taggingQueueScope === "selected" && taggingQueueFolderIds.length === 0)}
>
{taggerQueueing ? "Queueing..." : "Queue tagging"}
</button>
<button
className="rounded-md border border-amber-400/20 bg-amber-500/10 px-3 py-1.5 text-xs text-amber-200 transition-colors hover:bg-amber-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => runQueueAction("clear")}
disabled={taggerQueueing || taggerClearing || (taggingQueueScope === "selected" && taggingQueueFolderIds.length === 0)}
>
{taggerClearing ? "Clearing..." : "Clear queued jobs"}
</button>
</div>
</SettingsRow>
{taggerQueueStatus ? <p className="pt-4 text-xs text-gray-500">{taggerQueueStatus}</p> : null}
</SettingsCard>
<SettingsCard title="Captioning">
<div className="rounded-xl border border-dashed border-white/[0.09] bg-black/20 px-4 py-4">
<p className="text-sm font-medium text-white/50">Coming soon</p>
</div>
</SettingsCard>
</SectionShell>
) : null}
</div>
) : (
<div className="space-y-8">
<SectionShell
eyebrow="Workers"
title="Background processing"
>
<SettingsCard title="Queue summary" description="Live totals across all folder workers.">
<div className="grid gap-3 sm:grid-cols-3">
<div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
<p className="text-[11px] uppercase tracking-[0.14em] text-gray-600">Tagging queued</p>
<p className="mt-2 text-2xl font-semibold text-white">{totalQueuedJobs.toLocaleString()}</p>
</div>
<div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
<p className="text-[11px] uppercase tracking-[0.14em] text-gray-600">Selected folders</p>
<p className="mt-2 text-2xl font-semibold text-white">{selectedFolders.length.toLocaleString()}</p>
</div>
<div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
<p className="text-[11px] uppercase tracking-[0.14em] text-gray-600">Library folders</p>
<p className="mt-2 text-2xl font-semibold text-white">{folders.length.toLocaleString()}</p>
</div>
</div>
</SettingsCard>
<SettingsCard title="Worker controls" description="Pause, resume, retry, and per-folder failure details are available in the background tasks panel.">
<div className="flex items-center justify-between gap-4 rounded-xl border border-white/[0.07] bg-black/20 p-4">
<p className="text-sm text-gray-400">Workers are running in the background.</p>
<StatusPill tone="ready">Live</StatusPill>
</div>
</SettingsCard>
</SectionShell>
</div>
)}
</div>
</main>
</div>
+17
View File
@@ -138,6 +138,23 @@ export function Sidebar() {
Explore
</span>
</div>
<div
className={`flex items-center gap-2.5 px-3 py-2 rounded-lg cursor-pointer transition-all duration-150 ${
activeView === "duplicates"
? "bg-white/8 text-white"
: "text-gray-500 hover:text-gray-200 hover:bg-white/5"
}`}
onClick={() => setView("duplicates")}
>
<svg className="w-3.5 h-3.5 shrink-0" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5}
d="M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z" />
</svg>
<span className={`text-[13px] font-medium ${activeView === "duplicates" ? "text-white" : ""}`}>
Duplicates
</span>
</div>
</div>
{/* Section label */}
+318 -186
View File
@@ -1,242 +1,374 @@
import { useEffect } from "react";
import { useEffect, useLayoutEffect, useMemo, useRef, useState } from "react";
import { motion } from "framer-motion";
import { convertFileSrc } from "@tauri-apps/api/core";
import { useGalleryStore, TagCloudEntry } from "../store";
import { ExploreTagEntry, TagCloudEntry, useGalleryStore } from "../store";
// Accent glow colours for the hover ring — cycled by index
const GLOWS: string[] = [
"rgba(59,130,246,0.5)",
"rgba(168,85,247,0.5)",
"rgba(16,185,129,0.5)",
"rgba(245,158,11,0.5)",
"rgba(236,72,153,0.5)",
"rgba(6,182,212,0.5)",
"rgba(249,115,22,0.5)",
"rgba(34,197,94,0.5)",
const ACCENTS = [
"#60a5fa",
"#c084fc",
"#4ade80",
"#fbbf24",
"#f472b4",
"#2dd4bf",
"#fb923c",
"#a78bfa",
"#34d399",
"#f87171",
];
function pseudoRandom(seed: number): number {
const x = Math.sin(seed + 1) * 10000;
const GOLDEN_ANGLE = Math.PI * (3 - Math.sqrt(5));
function seeded(n: number): number {
const x = Math.sin(n * 9301 + 49297) * 233280;
return x - Math.floor(x);
}
// Map cluster size to a tile size bucket (px)
function getTileSize(count: number, maxCount: number): number {
if (maxCount === 0) return 72;
const ratio = count / maxCount;
if (ratio > 0.75) return 160;
if (ratio > 0.45) return 128;
if (ratio > 0.22) return 104;
if (ratio > 0.08) return 88;
return 72;
}
function TagButton({
entry,
index,
maxCount,
onSearch,
}: {
interface PlacedNode {
entry: TagCloudEntry;
index: number;
maxCount: number;
onSearch: (imageId: number) => void;
}) {
const size = getTileSize(entry.count, maxCount);
const glow = GLOWS[index % GLOWS.length];
x: number;
y: number;
w: number;
h: number;
accent: string;
driftX: number;
driftY: number;
driftDuration: number;
rotateSeed: number;
}
// Small random rotation for organic feel — larger tiles stay flatter
const maxRot = size >= 128 ? 0 : size >= 104 ? 3 : size >= 88 ? 6 : 10;
const rotation = (pseudoRandom(index * 7) - 0.5) * 2 * maxRot;
function buildCloud(entries: TagCloudEntry[], containerW: number, containerH: number): PlacedNode[] {
if (!entries.length || containerW <= 0 || containerH <= 0) return [];
const mt = Math.floor(pseudoRandom(index * 3) * 10) + 4;
const mr = Math.floor(pseudoRandom(index * 5) * 12) + 4;
const mb = Math.floor(pseudoRandom(index * 11) * 10) + 4;
const ml = Math.floor(pseudoRandom(index * 13) * 12) + 4;
const maxCount = Math.max(...entries.map((e) => e.count));
const cx = containerW / 2;
const cy = containerH / 2;
// Spread ellipse shrinks slightly to leave room for card half-widths at the edges
const spreadX = containerW * 0.42;
const spreadY = containerH * 0.36;
const n = entries.length;
const src = entry.thumbnail_path ? convertFileSrc(entry.thumbnail_path) : null;
// 1. Build initial positions using phyllotaxis spiral
const nodes: PlacedNode[] = entries.map((entry, i) => {
const ratio = Math.max(entry.count / maxCount, 0.08);
// Cards scale from 110px to 230px wide; height is 3/4 of width
const w = 110 + Math.sqrt(ratio) * 120;
const h = w * 0.75;
const radialRatio = Math.sqrt((i + 0.5) / n);
const angle = i * GOLDEN_ANGLE;
return {
entry,
index: i,
x: cx + Math.cos(angle) * radialRatio * spreadX,
y: cy + Math.sin(angle) * radialRatio * spreadY,
w,
h,
accent: ACCENTS[i % ACCENTS.length],
driftX: (seeded(i + 11) - 0.5) * 18,
driftY: (seeded(i + 17) - 0.5) * 14,
driftDuration: 8 + seeded(i + 23) * 7,
rotateSeed: (seeded(i + 31) - 0.5) * 4,
};
});
// 2. Iterative overlap resolution — no physics, just push apart
const PAD = 24;
for (let iter = 0; iter < 80; iter++) {
for (let a = 0; a < nodes.length; a++) {
const na = nodes[a];
for (let b = a + 1; b < nodes.length; b++) {
const nb = nodes[b];
const dx = nb.x - na.x;
const dy = nb.y - na.y;
const overlapX = (na.w + nb.w) / 2 + PAD - Math.abs(dx);
const overlapY = (na.h + nb.h) / 2 + PAD - Math.abs(dy);
if (overlapX <= 0 || overlapY <= 0) continue;
// Push along the smaller overlap axis
if (overlapX < overlapY) {
const push = overlapX * 0.5 * (dx >= 0 ? 1 : -1);
nb.x += push;
na.x -= push;
} else {
const push = overlapY * 0.5 * (dy >= 0 ? 1 : -1);
nb.y += push;
na.y -= push;
}
}
// Pull gently back toward anchor to prevent runaway drift
na.x += (cx + Math.cos(na.index * GOLDEN_ANGLE) * Math.sqrt((na.index + 0.5) / n) * spreadX - na.x) * 0.05;
na.y += (cy + Math.sin(na.index * GOLDEN_ANGLE) * Math.sqrt((na.index + 0.5) / n) * spreadY - na.y) * 0.05;
}
}
// 3. Clamp so cards never poke outside the container
return nodes.map((node) => ({
...node,
x: Math.min(Math.max(node.x, node.w / 2 + 16), containerW - node.w / 2 - 16),
y: Math.min(Math.max(node.y, node.h / 2 + 16), containerH - node.h / 2 - 16),
}));
}
function CloudCard({ node, onOpen }: { node: PlacedNode; onOpen: (imageIds: number[]) => void }) {
const src = node.entry.thumbnail_path ? convertFileSrc(node.entry.thumbnail_path) : null;
const { w, h, accent } = node;
return (
<motion.button
initial={{ opacity: 0, scale: 0.5, rotate: rotation * 2 }}
animate={{ opacity: 1, scale: 1, rotate: rotation }}
className="group absolute overflow-hidden rounded-2xl border border-white/8 bg-white/[0.04] text-left shadow-[0_8px_40px_rgba(0,0,0,0.5)] focus:outline-none focus-visible:ring-2 focus-visible:ring-white/30"
style={{ width: w, height: h, left: node.x - w / 2, top: node.y - h / 2 }}
initial={{ opacity: 0, scale: 0.75 }}
animate={{
opacity: 1,
scale: 1,
x: [0, node.driftX, 0],
y: [0, node.driftY, 0],
rotate: [node.rotateSeed, node.rotateSeed + 1.4, node.rotateSeed],
}}
transition={{
delay: index * 0.025,
type: "spring",
stiffness: 200,
damping: 18,
}}
whileHover={{
scale: 1.12,
rotate: 0,
transition: { type: "spring", stiffness: 400, damping: 22 },
}}
whileTap={{ scale: 0.92 }}
onClick={() => onSearch(entry.representative_image_id)}
title={`${entry.count} similar ${entry.count === 1 ? "photo" : "photos"}`}
style={{
width: size,
height: size,
margin: `${mt}px ${mr}px ${mb}px ${ml}px`,
borderRadius: 12,
border: "2px solid rgba(255,255,255,0.08)",
background: "rgba(255,255,255,0.04)",
cursor: "pointer",
padding: 0,
overflow: "hidden",
position: "relative",
flexShrink: 0,
boxShadow: "none",
transition: "border-color 0.15s, box-shadow 0.15s",
}}
onMouseEnter={(e) => {
const el = e.currentTarget;
el.style.borderColor = glow;
el.style.boxShadow = `0 0 16px ${glow}, 0 0 32px ${glow.replace("0.5", "0.25")}`;
}}
onMouseLeave={(e) => {
const el = e.currentTarget;
el.style.borderColor = "rgba(255,255,255,0.08)";
el.style.boxShadow = "none";
opacity: { duration: 0.3, delay: Math.min(node.index * 0.035, 0.7) },
scale: { duration: 0.3, delay: Math.min(node.index * 0.035, 0.7) },
x: { duration: node.driftDuration, repeat: Infinity, ease: "easeInOut", delay: seeded(node.index + 41) * 3 },
y: { duration: node.driftDuration + 1.6, repeat: Infinity, ease: "easeInOut", delay: seeded(node.index + 51) * 3 },
rotate: { duration: node.driftDuration + 0.9, repeat: Infinity, ease: "easeInOut" },
}}
whileHover={{ scale: 1.06, rotate: 0, transition: { duration: 0.18 } }}
onClick={() => onOpen(node.entry.image_ids)}
title={`Open cluster — ${node.entry.count.toLocaleString()} images`}
>
{src ? (
<img
src={src}
alt=""
className="absolute inset-0 h-full w-full object-cover transition-transform duration-500 group-hover:scale-105"
draggable={false}
style={{
width: "100%",
height: "100%",
objectFit: "cover",
display: "block",
}}
/>
) : (
// Fallback placeholder when no thumbnail exists yet
<div
style={{
width: "100%",
height: "100%",
background: "rgba(255,255,255,0.06)",
}}
/>
<div className="absolute inset-0 bg-gradient-to-br from-white/[0.07] to-transparent" />
)}
{/* Count badge — bottom-right corner */}
<div className="absolute inset-0 bg-gradient-to-t from-black/85 via-black/20 to-transparent" />
{/* Accent glow on hover */}
<div
style={{
position: "absolute",
bottom: 5,
right: 5,
background: "rgba(0,0,0,0.6)",
color: "rgba(255,255,255,0.85)",
fontSize: 10,
fontWeight: 600,
lineHeight: 1,
padding: "3px 6px",
borderRadius: 6,
backdropFilter: "blur(4px)",
pointerEvents: "none",
}}
className="absolute inset-0 opacity-0 transition-opacity duration-300 group-hover:opacity-100"
style={{ background: `radial-gradient(ellipse at bottom, ${accent}25, transparent 70%)` }}
/>
<div className="absolute inset-x-0 bottom-0 p-3">
<div className="mb-2 h-px rounded-full" style={{ background: `linear-gradient(to right, ${accent}80, transparent)` }} />
<div className="flex items-end justify-between gap-2">
<div>
<p className="text-[9px] uppercase tracking-[0.18em] text-white/35">Cluster</p>
<p className="text-base font-semibold leading-none text-white">{node.entry.count.toLocaleString()}</p>
</div>
<span
className="rounded-full border px-2 py-0.5 text-[9px] uppercase tracking-[0.1em] opacity-0 transition-opacity duration-200 group-hover:opacity-100"
style={{ borderColor: `${accent}50`, color: accent, backgroundColor: `${accent}12` }}
>
{entry.count}
Open
</span>
</div>
</div>
</motion.button>
);
}
// Actual tag cloud — word size driven by log-scaled frequency
function TagWord({
entry,
index,
logMin,
logRange,
onSearch,
}: {
entry: ExploreTagEntry;
index: number;
logMin: number;
logRange: number;
onSearch: (tag: string) => void;
}) {
const ratio = logRange > 0 ? (Math.log(Math.max(entry.count, 1)) - logMin) / logRange : 0.5;
const fontSize = 11 + ratio * 28; // 11px 39px
const accent = ACCENTS[index % ACCENTS.length];
const tilt = (seeded(index + 5) - 0.5) * 7;
return (
<motion.button
initial={{ opacity: 0, scale: 0.6 }}
animate={{ opacity: 0.4 + ratio * 0.6, scale: 1 }}
transition={{ delay: Math.min(index * 0.008, 0.55), duration: 0.22 }}
whileHover={{ scale: 1.2, opacity: 1, rotate: 0, transition: { duration: 0.14 } }}
className="group inline-flex items-center gap-1 rounded-full px-2 py-1 transition-colors hover:bg-white/[0.07]"
style={{ fontSize, rotate: tilt }}
onClick={() => onSearch(entry.tag)}
title={`${entry.tag}${entry.count.toLocaleString()} images`}
>
<span
className="font-medium leading-none"
style={{ color: ratio > 0.55 ? accent : "rgba(255,255,255,0.82)" }}
>
{entry.tag}
</span>
<span
className="rounded-full px-1.5 py-0.5 text-[9px] tabular-nums opacity-0 transition-opacity group-hover:opacity-100"
style={{ backgroundColor: `${accent}22`, color: accent }}
>
{entry.count.toLocaleString()}
</span>
</motion.button>
);
}
function Spinner() {
return (
<motion.div
className="h-5 w-5 rounded-full border-2 border-white/15 border-t-white/50"
animate={{ rotate: 360 }}
transition={{ duration: 0.85, repeat: Infinity, ease: "linear" }}
/>
);
}
// Separate component so its useLayoutEffect fires when the canvas is actually
// mounted — not at TagCloud mount time when the container may still be hidden
// behind a loading state.
function ClusterCloud({
entries,
onOpen,
}: {
entries: TagCloudEntry[];
onOpen: (imageIds: number[]) => void;
}) {
const canvasRef = useRef<HTMLDivElement>(null);
const [canvasSize, setCanvasSize] = useState({ w: 0, h: 0 });
useLayoutEffect(() => {
const el = canvasRef.current;
if (!el) return;
const update = () => {
const r = el.getBoundingClientRect();
setCanvasSize({ w: r.width, h: r.height });
};
update();
const ro = new ResizeObserver(update);
ro.observe(el);
return () => ro.disconnect();
}, []);
const nodes = useMemo(
() => buildCloud(entries, canvasSize.w, canvasSize.h),
[entries, canvasSize.w, canvasSize.h],
);
return (
<div ref={canvasRef} className="relative min-h-0 flex-1 overflow-hidden">
<div className="pointer-events-none absolute inset-0 opacity-[0.07] [background-image:radial-gradient(circle,rgba(255,255,255,0.6)_1px,transparent_1px)] [background-size:28px_28px]" />
{nodes.map((node) => (
<CloudCard key={`${node.entry.representative_image_id}:${node.index}`} node={node} onOpen={onOpen} />
))}
</div>
);
}
export function TagCloud() {
const exploreMode = useGalleryStore((state) => state.exploreMode);
const setExploreMode = useGalleryStore((state) => state.setExploreMode);
const tagCloudEntries = useGalleryStore((state) => state.tagCloudEntries);
const tagCloudLoading = useGalleryStore((state) => state.tagCloudLoading);
const loadTagCloud = useGalleryStore((state) => state.loadTagCloud);
const searchByTag = useGalleryStore((state) => state.searchByTag);
const exploreTagEntries = useGalleryStore((state) => state.exploreTagEntries);
const exploreTagLoading = useGalleryStore((state) => state.exploreTagLoading);
const loadExploreTags = useGalleryStore((state) => state.loadExploreTags);
const showVisualCluster = useGalleryStore((state) => state.showVisualCluster);
const searchForTag = useGalleryStore((state) => state.searchForTag);
const selectedFolderId = useGalleryStore((state) => state.selectedFolderId);
useEffect(() => {
void loadTagCloud();
}, [selectedFolderId]);
if (exploreMode === "visual") void loadTagCloud();
else void loadExploreTags();
}, [exploreMode, selectedFolderId, loadTagCloud, loadExploreTags]);
const maxCount =
tagCloudEntries.length > 0
? Math.max(...tagCloudEntries.map((e) => e.count))
: 1;
const { logMin, logRange } = useMemo(() => {
if (!exploreTagEntries.length) return { logMin: 0, logRange: 1 };
const logs = exploreTagEntries.map((e) => Math.log(Math.max(e.count, 1)));
const lo = Math.min(...logs);
const hi = Math.max(...logs);
return { logMin: lo, logRange: hi - lo || 1 };
}, [exploreTagEntries]);
const loading = exploreMode === "visual" ? tagCloudLoading : exploreTagLoading;
const hasEntries = exploreMode === "visual" ? tagCloudEntries.length > 0 : exploreTagEntries.length > 0;
const entryCount = exploreMode === "visual" ? tagCloudEntries.length : exploreTagEntries.length;
return (
<div className="flex-1 flex flex-col items-center min-h-0 overflow-y-auto">
<div className="flex min-h-0 flex-1 flex-col overflow-hidden bg-[radial-gradient(ellipse_at_top,rgba(59,130,246,0.08),transparent_50%),radial-gradient(ellipse_at_80%_75%,rgba(168,85,247,0.07),transparent_40%),#07080f]">
{/* Header */}
<motion.div
className="text-center pt-14 pb-8 shrink-0"
initial={{ opacity: 0, y: -10 }}
animate={{ opacity: 1, y: 0 }}
transition={{ duration: 0.35 }}
>
<h2 className="text-[22px] font-semibold text-white/70 tracking-tight mb-2">
Explore your library
</h2>
<p className="text-[13px] text-white/25">
Visual clusters from your photos sized by how many match
</p>
</motion.div>
{/* Loading */}
{tagCloudLoading && (
<div className="flex-1 flex flex-col items-center justify-center gap-4">
<motion.svg
className="w-8 h-8 text-white/20"
viewBox="0 0 24 24"
fill="none"
animate={{ rotate: 360 }}
transition={{ duration: 1, repeat: Infinity, ease: "linear" }}
>
<circle
cx="12"
cy="12"
r="10"
stroke="currentColor"
strokeWidth="2"
strokeOpacity="0.2"
/>
<path
d="M4 12a8 8 0 018-8"
stroke="currentColor"
strokeWidth="2"
strokeLinecap="round"
/>
</motion.svg>
<p className="text-[12px] text-white/20">Clustering your library</p>
</div>
)}
{/* Empty state */}
{!tagCloudLoading && tagCloudEntries.length === 0 && (
<div className="flex-1 flex items-center justify-center">
<p className="text-[13px] text-white/25 text-center max-w-xs leading-relaxed">
No embeddings yet. Add a folder and wait for the embedding worker to
finish, then come back here.
<div className="shrink-0 border-b border-white/[0.05] px-6 py-4">
<div className="flex items-center justify-between gap-4">
<div className="min-w-0">
<h2 className="text-[15px] font-semibold text-white">Explore</h2>
<p className="mt-0.5 truncate text-[11px] text-white/30">
{loading
? exploreMode === "visual" ? "Computing visual clusters…" : "Loading tags…"
: hasEntries
? exploreMode === "visual"
? `${entryCount} cluster${entryCount !== 1 ? "s" : ""} — click any to open`
: `${entryCount} tag${entryCount !== 1 ? "s" : ""} — click any to search`
: exploreMode === "visual"
? "No clusters — images need embeddings first"
: "No tags — run the AI tagger or add tags manually"}
</p>
</div>
)}
<div className="flex shrink-0 rounded-lg border border-white/8 bg-white/[0.03] p-0.5">
<button
className={`rounded-md px-3 py-1.5 text-xs transition-colors ${
exploreMode === "visual" ? "bg-white/10 text-white" : "text-gray-500 hover:text-gray-300"
}`}
onClick={() => setExploreMode("visual")}
>
Clusters
</button>
<button
className={`rounded-md px-3 py-1.5 text-xs transition-colors ${
exploreMode === "tags" ? "bg-white/10 text-white" : "text-gray-500 hover:text-gray-300"
}`}
onClick={() => setExploreMode("tags")}
>
Tag Cloud
</button>
</div>
</div>
</div>
{/* Cluster grid */}
{!tagCloudLoading && tagCloudEntries.length > 0 && (
<div className="flex flex-wrap justify-center px-12 pb-16 max-w-5xl w-full">
{tagCloudEntries.map((entry, index) => (
<TagButton
key={entry.representative_image_id}
{loading ? (
<div className="flex flex-1 items-center justify-center gap-3 text-white/25">
<Spinner />
<span className="text-sm">{exploreMode === "visual" ? "Computing clusters…" : "Loading tags…"}</span>
</div>
) : !hasEntries ? (
<div className="flex flex-1 items-center justify-center px-8">
<p className="max-w-xs text-center text-sm leading-relaxed text-white/25">
{exploreMode === "visual"
? "No visual clusters yet. Images need embeddings before they can be grouped. Check indexing progress in the sidebar."
: "No tags yet. Run the AI tagger from Settings, or add tags manually in the image preview."}
</p>
</div>
) : exploreMode === "visual" ? (
<ClusterCloud entries={tagCloudEntries} onOpen={showVisualCluster} />
) : (
/* Tag cloud — words sized by log-scaled frequency, wrapped freely */
<div className="overflow-y-auto px-8 py-8">
<div className="flex flex-wrap items-center justify-center gap-x-0.5 gap-y-1 leading-none">
{exploreTagEntries.map((entry, index) => (
<TagWord
key={entry.tag}
entry={entry}
index={index}
maxCount={maxCount}
onSearch={searchByTag}
logMin={logMin}
logRange={logRange}
onSearch={searchForTag}
/>
))}
</div>
)}
{!tagCloudLoading && tagCloudEntries.length > 0 && (
<p className="shrink-0 pb-8 text-[11px] text-white/12 text-center">
Grouped by visual similarity · CLIP ViT-B/32
</p>
</div>
)}
</div>
);
+189 -51
View File
@@ -1,11 +1,14 @@
import { useEffect, useRef, useState } from "react";
import { tileSizeForZoom, useGalleryStore, SortOrder, MediaFilter, SearchMode } from "../store";
import { invoke } from "@tauri-apps/api/core";
import { tileSizeForZoom, useGalleryStore, SortOrder, MediaFilter, SearchCommand, parseSearchValue, searchModeLabel, ExploreTagEntry } from "../store";
const BASE_SORT_OPTIONS: { value: SortOrder; label: string }[] = [
{ value: "date_desc", label: "Newest first" },
{ value: "date_asc", label: "Oldest first" },
{ value: "name_asc", label: "Name AZ" },
{ value: "name_desc", label: "Name ZA" },
{ value: "rating_desc", label: "Highest rated" },
{ value: "rating_asc", label: "Lowest rated" },
{ value: "size_desc", label: "Largest first" },
{ value: "size_asc", label: "Smallest first" },
];
@@ -116,12 +119,27 @@ function FilterPill({
);
}
function commandPrefix(command: SearchCommand | null): string | null {
switch (command) {
case "semantic":
return "/s";
case "tag":
return "/t";
default:
return null;
}
}
function composeSearchValue(command: SearchCommand | null, query: string): string {
const prefix = commandPrefix(command);
if (!prefix) return query;
return query.length > 0 ? `${prefix} ${query}` : prefix;
}
export function Toolbar() {
const search = useGalleryStore((state) => state.search);
const setSearch = useGalleryStore((state) => state.setSearch);
const clearSearch = useGalleryStore((state) => state.clearSearch);
const searchMode = useGalleryStore((state) => state.searchMode);
const setSearchMode = useGalleryStore((state) => state.setSearchMode);
const sort = useGalleryStore((state) => state.sort);
const setSort = useGalleryStore((state) => state.setSort);
const totalImages = useGalleryStore((state) => state.totalImages);
@@ -133,20 +151,26 @@ export function Toolbar() {
const setMediaFilter = useGalleryStore((state) => state.setMediaFilter);
const favoritesOnly = useGalleryStore((state) => state.favoritesOnly);
const setFavoritesOnly = useGalleryStore((state) => state.setFavoritesOnly);
const minimumRating = useGalleryStore((state) => state.minimumRating);
const setMinimumRating = useGalleryStore((state) => state.setMinimumRating);
const failedEmbeddingsOnly = useGalleryStore((state) => state.failedEmbeddingsOnly);
const setFailedEmbeddingsOnly = useGalleryStore((state) => state.setFailedEmbeddingsOnly);
const similarScope = useGalleryStore((state) => state.similarScope);
const setSimilarScope = useGalleryStore((state) => state.setSimilarScope);
const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress);
const zoomPreset = useGalleryStore((state) => state.zoomPreset);
const setZoomPreset = useGalleryStore((state) => state.setZoomPreset);
const hasAnyFailedEmbeddings = Object.values(mediaJobProgress).some((p) => p.embedding_failed > 0);
const [searchValue, setSearchValue] = useState(search);
const [searchCommand, setSearchCommand] = useState<SearchCommand | null>(null);
const [searchQuery, setSearchQuery] = useState(search);
const [searchPanelOpen, setSearchPanelOpen] = useState(false);
const [tagSuggestions, setTagSuggestions] = useState<ExploreTagEntry[]>([]);
const debounceRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const suggestDebounceRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const searchInputRef = useRef<HTMLInputElement>(null);
// Tracks whether the user has typed in the search box at least once.
// Prevents the debounce effect from dispatching setSearch on initial mount
// when searchValue === search (which would wipe a loadSimilarImages result).
const searchShellRef = useRef<HTMLDivElement>(null);
const userHasTyped = useRef(false);
const selectedFolder = folders.find((folder) => folder.id === selectedFolderId);
@@ -154,11 +178,8 @@ export function Toolbar() {
const tileSize = tileSizeForZoom(zoomPreset);
const sortOptions = getSortOptions(mediaFilter);
const hasActiveSearch = search.trim().length > 0;
const searchModes: { value: SearchMode; label: string }[] = [
{ value: "filename", label: "Filename" },
{ value: "semantic", label: "Semantic" },
];
const parsedSearch = parseSearchValue(composeSearchValue(searchCommand, searchQuery));
const isSimilarResults = collectionTitle === "Similar Images";
// If current sort is video-only but we switched away from video filter, reset to date_desc
useEffect(() => {
@@ -170,35 +191,62 @@ export function Toolbar() {
useEffect(() => {
if (!userHasTyped.current) return;
if (debounceRef.current) clearTimeout(debounceRef.current);
debounceRef.current = setTimeout(() => { setSearch(searchValue); }, 200);
debounceRef.current = setTimeout(() => { setSearch(composeSearchValue(searchCommand, searchQuery)); }, 200);
return () => { if (debounceRef.current) clearTimeout(debounceRef.current); };
}, [searchValue, setSearch]);
}, [searchCommand, searchQuery, setSearch]);
useEffect(() => {
setSearchValue(search);
const parsed = parseSearchValue(search);
setSearchCommand(parsed.prefix && parsed.mode !== "filename" ? parsed.mode : null);
setSearchQuery(parsed.prefix ? parsed.query : search);
}, [search]);
// Fetch tag suggestions when in tag mode
useEffect(() => {
if (searchCommand !== "tag") {
setTagSuggestions([]);
return;
}
if (suggestDebounceRef.current) clearTimeout(suggestDebounceRef.current);
suggestDebounceRef.current = setTimeout(async () => {
try {
const results = await invoke<ExploreTagEntry[]>("search_tags_autocomplete", {
params: { query: searchQuery.trim(), folder_id: selectedFolderId ?? null, limit: 10 },
});
setTagSuggestions(results);
} catch {
setTagSuggestions([]);
}
}, 120);
return () => { if (suggestDebounceRef.current) clearTimeout(suggestDebounceRef.current); };
}, [searchCommand, searchQuery, selectedFolderId]);
useEffect(() => {
const handleKeyDown = (event: KeyboardEvent) => {
const isModeToggle = (event.ctrlKey || event.metaKey) && event.shiftKey && event.key.toLowerCase() === "s";
if (isModeToggle) {
event.preventDefault();
setSearchMode(searchMode === "semantic" ? "filename" : "semantic");
searchInputRef.current?.focus();
return;
}
const activeElement = document.activeElement;
const searchFocused = activeElement === searchInputRef.current;
if (event.key === "Escape" && (searchFocused || hasActiveSearch)) {
event.preventDefault();
setSearchValue("");
setSearchCommand(null);
setSearchQuery("");
clearSearch();
}
};
window.addEventListener("keydown", handleKeyDown);
return () => window.removeEventListener("keydown", handleKeyDown);
}, [clearSearch, hasActiveSearch, searchMode, setSearchMode]);
}, [clearSearch, hasActiveSearch]);
useEffect(() => {
const close = (event: PointerEvent) => {
if (searchShellRef.current?.contains(event.target as Node)) return;
setSearchPanelOpen(false);
};
window.addEventListener("pointerdown", close);
return () => window.removeEventListener("pointerdown", close);
}, []);
const showTagSuggestions = searchCommand === "tag" && searchPanelOpen;
const showCommandHints = !searchCommand && searchPanelOpen;
return (
<div className="relative z-20 shrink-0 border-b border-white/[0.06] bg-gray-950/80 backdrop-blur-xl">
@@ -212,9 +260,9 @@ export function Toolbar() {
? `${loadedCount.toLocaleString()} / ${totalImages.toLocaleString()}`
: totalImages.toLocaleString()}
</span>
{(hasActiveSearch || searchMode === "semantic") && (
{hasActiveSearch && (
<span className="rounded-md border border-blue-500/20 bg-blue-500/10 px-2 py-0.5 text-[10px] uppercase tracking-[0.14em] text-blue-300 shrink-0">
{searchMode === "semantic" ? "Semantic Search" : "Filename Search"}
{searchModeLabel(parsedSearch.mode)}
</span>
)}
</div>
@@ -222,23 +270,8 @@ export function Toolbar() {
<div className="flex-1" />
{/* Search */}
<div ref={searchShellRef} className="relative">
<div className="flex items-center rounded-lg border border-white/8 bg-white/5 overflow-hidden">
<div className="flex items-center pl-2 pr-1 gap-1 border-r border-white/8">
{searchModes.map((mode) => (
<button
key={mode.value}
className={`rounded-md px-2 py-1 text-[11px] transition-colors ${
searchMode === mode.value
? "bg-white/10 text-white"
: "text-gray-500 hover:text-gray-200"
}`}
title={mode.value === "semantic" ? "Toggle with Ctrl/Cmd+Shift+S" : "Toggle with Ctrl/Cmd+Shift+S"}
onClick={() => setSearchMode(mode.value)}
>
{mode.label}
</button>
))}
</div>
<div className="relative">
<svg className="pointer-events-none absolute left-2.5 top-1/2 h-3.5 w-3.5 -translate-y-1/2 text-gray-600"
fill="none" viewBox="0 0 24 24" stroke="currentColor">
@@ -248,20 +281,50 @@ export function Toolbar() {
<input
ref={searchInputRef}
type="text"
value={searchValue}
value={searchQuery}
onChange={(event) => {
userHasTyped.current = true;
setSearchValue(event.target.value);
const nextValue = event.target.value;
if (!searchCommand) {
const parsed = parseSearchValue(nextValue);
if (parsed.prefix) {
setSearchCommand(parsed.mode === "filename" ? null : parsed.mode);
setSearchQuery(parsed.query);
return;
}
}
setSearchQuery(nextValue);
}}
placeholder={searchMode === "semantic" ? "Search by meaning..." : "Search filenames..."}
className="w-64 bg-transparent py-1.5 pl-8 pr-9 text-sm text-white placeholder:text-gray-600 focus:outline-none transition-colors"
onKeyDown={(event) => {
if (event.key === "Backspace" && searchQuery.length === 0 && searchCommand !== null) {
event.preventDefault();
setSearchCommand(null);
}
}}
onFocus={() => setSearchPanelOpen(true)}
placeholder="Search files, or use /s /t"
className={`w-64 bg-transparent py-1.5 pr-9 text-sm text-white placeholder:text-gray-600 focus:outline-none transition-colors ${searchCommand !== null ? "pl-16" : "pl-8"}`}
/>
{searchValue.trim().length > 0 && (
{searchCommand !== null ? (
<div className="absolute left-8 top-1/2 -translate-y-1/2">
<button
type="button"
className="rounded-md border border-white/10 bg-white/[0.05] px-1.5 py-0.5 font-mono text-[11px] text-gray-300 transition-colors hover:bg-white/[0.08] hover:text-white"
onClick={() => { setSearchCommand(null); setTagSuggestions([]); }}
title="Remove search command"
>
{commandPrefix(searchCommand)}
</button>
</div>
) : null}
{searchQuery.trim().length > 0 || searchCommand !== null ? (
<button
className="absolute right-2 top-1/2 -translate-y-1/2 rounded-md p-1 text-gray-500 hover:bg-white/5 hover:text-gray-200 transition-colors"
title="Clear search"
onClick={() => {
setSearchValue("");
setSearchCommand(null);
setSearchQuery("");
setTagSuggestions([]);
clearSearch();
}}
>
@@ -269,10 +332,80 @@ export function Toolbar() {
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
)}
) : null}
</div>
</div>
{/* Tag autocomplete suggestions */}
{showTagSuggestions && tagSuggestions.length > 0 ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-72 rounded-xl border border-white/10 bg-gray-950/98 py-1 shadow-2xl backdrop-blur">
{tagSuggestions.map((entry) => (
<button
key={entry.tag}
className="flex w-full items-center justify-between gap-3 px-3 py-2 text-left transition-colors hover:bg-white/[0.05]"
onMouseDown={(e) => {
// mousedown fires before input blur, so we prevent losing focus
e.preventDefault();
userHasTyped.current = true;
setSearchQuery(entry.tag);
setSearch(`/t ${entry.tag}`);
setSearchPanelOpen(false);
searchInputRef.current?.blur();
}}
>
<span className="text-sm text-white/88">{entry.tag}</span>
<span className="shrink-0 text-[11px] tabular-nums text-white/30">{entry.count.toLocaleString()}</span>
</button>
))}
</div>
) : null}
{/* Tag mode with no suggestions yet — show a brief hint */}
{showTagSuggestions && tagSuggestions.length === 0 && searchQuery.trim().length > 0 ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-72 rounded-xl border border-white/10 bg-gray-950/98 px-3 py-2.5 shadow-2xl backdrop-blur">
<p className="text-xs text-white/25">No matching tags</p>
</div>
) : null}
{/* Semantic mode hint */}
{searchCommand === "semantic" && searchPanelOpen ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-72 rounded-xl border border-white/10 bg-gray-950/98 px-3 py-2.5 shadow-2xl backdrop-blur">
<p className="text-xs text-white/40">Search by meaning and visual concepts</p>
</div>
) : null}
{/* Command hints — only shown when no command is active */}
{showCommandHints ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-64 rounded-xl border border-white/10 bg-gray-950/98 py-1 shadow-2xl backdrop-blur">
{(
[
{ command: "tag" as SearchCommand, prefix: "/t", label: "Tags", description: "Search AI and user tags" },
{ command: "semantic" as SearchCommand, prefix: "/s", label: "Semantic", description: "Search by meaning" },
] as const
).map((option) => (
<button
key={option.prefix}
className="flex w-full items-center gap-3 px-3 py-2 text-left transition-colors hover:bg-white/[0.05]"
onMouseDown={(e) => {
e.preventDefault();
userHasTyped.current = true;
setSearchCommand(option.command);
searchInputRef.current?.focus();
}}
>
<span className="rounded border border-white/10 bg-white/[0.04] px-1.5 py-0.5 font-mono text-[11px] text-gray-400">
{option.prefix}
</span>
<div>
<p className="text-sm text-gray-200">{option.label}</p>
<p className="text-xs text-gray-500">{option.description}</p>
</div>
</button>
))}
</div>
) : null}
</div>
{/* Sort */}
<SortDropdown value={sort} onChange={setSort} options={sortOptions} />
@@ -302,10 +435,14 @@ export function Toolbar() {
{/* Filter row */}
<div className="flex items-center gap-1 px-4 pb-1.5">
<FilterPill label="All" active={mediaFilter === "all" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("all"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="All" active={mediaFilter === "all" && !favoritesOnly && !failedEmbeddingsOnly && minimumRating === 0} onClick={() => { setMediaFilter("all"); setFavoritesOnly(false); setMinimumRating(0); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Images" active={mediaFilter === "image" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("image"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Videos" active={mediaFilter === "video" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("video"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Favorites" active={favoritesOnly} onClick={() => { setFavoritesOnly(!favoritesOnly); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Rated" active={minimumRating === 1} onClick={() => { setMinimumRating(minimumRating === 1 ? 0 : 1); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="4★+" active={minimumRating === 4} onClick={() => { setMinimumRating(minimumRating === 4 ? 0 : 4); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Similar: Folder" active={similarScope === "current_folder"} onClick={() => setSimilarScope("current_folder")} />
<FilterPill label="Similar: All" active={similarScope === "all_media"} onClick={() => setSimilarScope("all_media")} />
{hasAnyFailedEmbeddings ? (
<FilterPill
label="Failed Embeddings"
@@ -314,6 +451,7 @@ export function Toolbar() {
onClick={() => setFailedEmbeddingsOnly(!failedEmbeddingsOnly)}
/>
) : null}
{isSimilarResults ? <span className="ml-2 text-[11px] text-gray-500">Current similar scope: {similarScope === "current_folder" ? "current folder" : "all media"}</span> : null}
</div>
</div>
);
+539 -31
View File
@@ -15,10 +15,14 @@ export type MediaKind = "image" | "video";
export type MediaFilter = "all" | MediaKind;
export type ZoomPreset = "compact" | "comfortable" | "detail";
export type SearchMode = "filename" | "semantic";
export type SearchCommand = "filename" | "semantic" | "tag";
export type CaptionAcceleration = "auto" | "cpu" | "directml";
export type CaptionDetail = "short" | "detailed" | "paragraph";
export type TaggerAcceleration = "auto" | "cpu" | "directml";
export type AiRating = "general" | "sensitive" | "questionable" | "explicit";
export type TaggingQueueScope = "all" | "selected";
export type SimilarScope = "all_media" | "current_folder";
export type ExploreMode = "visual" | "tags";
export interface ImageRecord {
id: number;
@@ -115,12 +119,33 @@ export interface ThumbnailBatch {
images: ImageRecord[];
}
export type ActiveView = "gallery" | "explore";
export type ActiveView = "gallery" | "explore" | "duplicates";
export interface TagCloudEntry {
count: number;
representative_image_id: number;
thumbnail_path: string | null;
image_ids: number[];
}
export interface ExploreTagEntry {
tag: string;
count: number;
representative_image_id: number;
thumbnail_path: string | null;
}
export interface DuplicateGroup {
file_hash: string;
file_size: number;
images: ImageRecord[];
}
export interface SimilarImagesPage {
images: ImageRecord[];
offset: number;
limit: number;
has_more: boolean;
}
export interface CaptionModelStatus {
@@ -171,6 +196,12 @@ export interface TaggerRuntimeProbe {
session: TaggerRuntimeSessionProbe;
}
export interface ParsedSearch {
mode: SearchCommand;
query: string;
prefix: string | null;
}
export type SortOrder =
| "date_desc"
| "date_asc"
@@ -178,6 +209,8 @@ export type SortOrder =
| "name_desc"
| "size_desc"
| "size_asc"
| "rating_desc"
| "rating_asc"
| "duration_desc"
| "duration_asc";
@@ -194,17 +227,25 @@ interface GalleryState {
sort: SortOrder;
mediaFilter: MediaFilter;
favoritesOnly: boolean;
minimumRating: number;
failedEmbeddingsOnly: boolean;
zoomPreset: ZoomPreset;
selectedImage: ImageRecord | null;
collectionTitle: string | null;
similarSourceImageId: number | null;
similarSourceFolderId: number | null;
similarHasMore: boolean;
similarScope: SimilarScope;
similarFolderId: number | null;
galleryScrollResetKey: number;
activeView: ActiveView;
exploreMode: ExploreMode;
tagCloudEntries: TagCloudEntry[];
tagCloudLoading: boolean;
tagCloudFolderId: number | null | undefined; // undefined = never loaded
exploreTagEntries: ExploreTagEntry[];
exploreTagLoading: boolean;
exploreTagsFolderId: number | null | undefined;
indexingProgress: Record<number, IndexProgress>;
mediaJobProgress: Record<number, FolderJobProgress>;
cacheDir: string;
@@ -218,6 +259,8 @@ interface GalleryState {
captionDetail: CaptionDetail;
aiCaptionsEnabled: boolean;
settingsOpen: boolean;
taggingQueueScope: TaggingQueueScope;
taggingQueueFolderIds: number[];
taggerModelStatus: TaggerModelStatus | null;
taggerModelPreparing: boolean;
@@ -225,9 +268,16 @@ interface GalleryState {
taggerModelProgress: TaggerModelProgress | null;
taggerAcceleration: TaggerAcceleration;
taggerThreshold: number;
taggerBatchSize: number;
taggerRuntimeProbe: TaggerRuntimeProbe | null;
taggerRuntimeChecking: boolean;
duplicateGroups: DuplicateGroup[];
duplicateScanning: boolean;
duplicateScanProgress: { scanned: number; total: number } | null;
duplicateSelectedIds: Set<number>;
duplicateLastScanned: number | null; // Unix timestamp (seconds)
loadFolders: () => Promise<void>;
loadBackgroundJobProgress: () => Promise<void>;
addFolder: (path: string) => Promise<void>;
@@ -243,14 +293,20 @@ interface GalleryState {
setSort: (sort: SortOrder) => void;
setMediaFilter: (filter: MediaFilter) => void;
setFavoritesOnly: (favoritesOnly: boolean) => void;
setMinimumRating: (minimumRating: number) => void;
setFailedEmbeddingsOnly: (failedEmbeddingsOnly: boolean) => void;
setZoomPreset: (zoomPreset: ZoomPreset) => void;
openImage: (image: ImageRecord) => void;
closeImage: () => void;
setView: (view: ActiveView) => void;
setExploreMode: (mode: ExploreMode) => void;
loadTagCloud: () => Promise<void>;
searchByTag: (imageId: number) => void;
loadSimilarImages: (imageId: number, folderId?: number | null, reset?: boolean) => Promise<void>;
loadExploreTags: () => Promise<void>;
showVisualCluster: (imageIds: number[]) => Promise<void>;
searchForTag: (tag: string) => void;
loadSimilarImages: (imageId: number, folderId?: number | null, reset?: boolean, sourceFolderId?: number | null) => Promise<void>;
loadSimilarByRegion: (imageId: number, crop: { x: number; y: number; w: number; h: number }, folderId?: number | null, sourceFolderId?: number | null) => Promise<void>;
setSimilarScope: (scope: SimilarScope) => void;
suggestImageTags: (imageId: number) => Promise<string[]>;
loadCaptionModelStatus: () => Promise<void>;
prepareCaptionModel: () => Promise<void>;
@@ -268,6 +324,9 @@ interface GalleryState {
setCaptionDetail: (detail: CaptionDetail) => Promise<void>;
setAiCaptionsEnabled: (enabled: boolean) => void;
setSettingsOpen: (open: boolean) => void;
setTaggingQueueScope: (scope: TaggingQueueScope) => void;
toggleTaggingQueueFolder: (folderId: number) => void;
setTaggingQueueFolderIds: (folderIds: number[]) => void;
retryFailedEmbeddings: (folderId: number) => Promise<void>;
updateImageDetails: (imageId: number, updates: { favorite?: boolean; rating?: number }) => Promise<void>;
setCacheDir: (dir: string) => void;
@@ -280,10 +339,21 @@ interface GalleryState {
setTaggerAcceleration: (acceleration: TaggerAcceleration) => Promise<void>;
loadTaggerThreshold: () => Promise<void>;
setTaggerThreshold: (threshold: number) => Promise<void>;
loadTaggerBatchSize: () => Promise<void>;
setTaggerBatchSize: (batchSize: number) => Promise<void>;
probeTaggerRuntime: () => Promise<void>;
queueTaggingJobs: (folderId?: number | null) => Promise<number>;
queueTaggingJobsForFolders: (folderIds: number[]) => Promise<number>;
queueTaggingForImage: (imageId: number) => Promise<number>;
clearTaggingJobs: (folderId?: number | null) => Promise<number>;
clearTaggingJobsForFolders: (folderIds: number[]) => Promise<number>;
loadDuplicateScanCache: (folderId?: number | null) => Promise<void>;
scanDuplicates: (folderId?: number | null) => Promise<void>;
toggleDuplicateSelected: (imageId: number) => void;
selectAllDuplicates: (imageIds: number[]) => void;
selectKeepFirstAllGroups: () => void;
clearDuplicateSelection: () => void;
deleteSelectedDuplicates: () => Promise<number>;
getImageTags: (imageId: number) => Promise<ImageTag[]>;
addUserTag: (imageId: number, tag: string) => Promise<ImageTag>;
removeTag: (tagId: number) => Promise<void>;
@@ -291,6 +361,12 @@ interface GalleryState {
const PAGE_SIZE = 200;
const AI_CAPTIONS_ENABLED_KEY = "phokus.aiCaptionsEnabled";
const SIMILAR_DISTANCE_THRESHOLD = 0.24;
let galleryRequestToken = 0;
let similarRequestToken = 0;
let tagCloudRequestToken = 0;
let exploreTagRequestToken = 0;
function initialAiCaptionsEnabled(): boolean {
if (typeof window === "undefined") return false;
@@ -312,19 +388,71 @@ function matchesSearch(image: ImageRecord, search: string): boolean {
return image.filename.toLowerCase().includes(search.toLowerCase());
}
function isDerivedCollectionTitle(collectionTitle: string | null): boolean {
return collectionTitle !== null;
}
export function parseSearchValue(search: string): ParsedSearch {
if (!search.trim()) {
return { mode: "filename", query: "", prefix: null };
}
const slashPrefix = search.match(/^\/([a-z])(?:\s|$)/i);
if (slashPrefix) {
const rawPrefix = slashPrefix[1].toLowerCase();
const query = search.length > 3 ? search.slice(3) : "";
if (rawPrefix === "s") {
return { mode: "semantic", query, prefix: "/s" };
}
if (rawPrefix === "t") {
return { mode: "tag", query, prefix: "/t" };
}
return { mode: "filename", query, prefix: rawPrefix === "f" ? "/f" : null };
}
const trimmed = search.trim();
const match = trimmed.match(/^([a-z]):\s*(.*)$/i);
if (!match) {
return { mode: "filename", query: trimmed, prefix: null };
}
const rawPrefix = match[1].toLowerCase();
const query = match[2].trim();
if (rawPrefix === "s") {
return { mode: "semantic", query, prefix: "s:" };
}
if (rawPrefix === "t") {
return { mode: "tag", query, prefix: "t:" };
}
return { mode: "filename", query, prefix: rawPrefix === "f" ? "f:" : null };
}
export function searchModeLabel(mode: SearchCommand): string {
switch (mode) {
case "semantic":
return "Semantic Search";
case "tag":
return "Tag Search";
default:
return "Filename Search";
}
}
function matchesFilters(
image: ImageRecord,
selectedFolderId: number | null,
mediaFilter: MediaFilter,
favoritesOnly: boolean,
minimumRating: number,
failedEmbeddingsOnly: boolean,
search: string,
): boolean {
const matchesFolder = selectedFolderId === null || image.folder_id === selectedFolderId;
const matchesMedia = mediaFilter === "all" || image.media_kind === mediaFilter;
const matchesFavorite = !favoritesOnly || image.favorite;
const matchesRating = image.rating >= minimumRating;
const matchesFailedEmbedding = !failedEmbeddingsOnly || image.embedding_status === "failed";
return matchesFolder && matchesMedia && matchesFavorite && matchesFailedEmbedding && matchesSearch(image, search);
return matchesFolder && matchesMedia && matchesFavorite && matchesRating && matchesFailedEmbedding && matchesSearch(image, search);
}
function compareNullableNumber(a: number | null, b: number | null): number {
@@ -349,6 +477,10 @@ function compareImages(a: ImageRecord, b: ImageRecord, sort: SortOrder): number
return compareNullableNumber(a.file_size, b.file_size);
case "size_desc":
return compareNullableNumber(b.file_size, a.file_size);
case "rating_asc":
return compareNullableNumber(a.rating, b.rating);
case "rating_desc":
return compareNullableNumber(b.rating, a.rating);
case "duration_asc":
return compareNullableNumber(a.duration_ms, b.duration_ms);
case "duration_desc":
@@ -424,17 +556,25 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
sort: "date_desc",
mediaFilter: "all",
favoritesOnly: false,
minimumRating: 0,
failedEmbeddingsOnly: false,
zoomPreset: "comfortable",
selectedImage: null,
collectionTitle: null,
similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false,
similarScope: "all_media",
similarFolderId: null,
galleryScrollResetKey: 0,
activeView: "gallery",
exploreMode: "visual",
tagCloudEntries: [],
tagCloudLoading: false,
tagCloudFolderId: undefined,
exploreTagEntries: [],
exploreTagLoading: false,
exploreTagsFolderId: undefined,
indexingProgress: {},
mediaJobProgress: {},
cacheDir: "",
@@ -448,6 +588,8 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
captionDetail: "paragraph",
aiCaptionsEnabled: initialAiCaptionsEnabled(),
settingsOpen: false,
taggingQueueScope: "all",
taggingQueueFolderIds: [],
taggerModelStatus: null,
taggerModelPreparing: false,
@@ -455,14 +597,33 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
taggerModelProgress: null,
taggerAcceleration: "auto",
taggerThreshold: 0.35,
taggerBatchSize: 8,
taggerRuntimeProbe: null,
taggerRuntimeChecking: false,
duplicateGroups: [],
duplicateScanning: false,
duplicateScanProgress: null,
duplicateSelectedIds: new Set(),
duplicateLastScanned: null,
setCacheDir: (cacheDir) => set({ cacheDir }),
loadFolders: async () => {
const folders = await invoke<Folder[]>("get_folders");
set({ folders });
set((state) => {
const folderIds = new Set(folders.map((folder) => folder.id));
const nextSelected = state.taggingQueueFolderIds.filter((folderId) => folderIds.has(folderId));
return {
folders,
taggingQueueFolderIds:
nextSelected.length > 0
? nextSelected
: state.taggingQueueScope === "selected" && folders.length > 0
? [folders[0].id]
: nextSelected,
};
});
},
loadBackgroundJobProgress: async () => {
@@ -507,29 +668,64 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
},
loadImages: async (reset = false) => {
const { selectedFolderId, search, searchMode, sort, loadedCount, mediaFilter, favoritesOnly, failedEmbeddingsOnly } = get();
const { selectedFolderId, search, sort, loadedCount, mediaFilter, favoritesOnly, minimumRating, failedEmbeddingsOnly } = get();
const parsedSearch = parseSearchValue(search);
const requestToken = ++galleryRequestToken;
set({ loadingImages: true, imageLoadError: null });
try {
if (searchMode === "semantic" && search.trim()) {
if (parsedSearch.mode === "semantic" && parsedSearch.query) {
const images = await invoke<ImageRecord[]>("semantic_search_images", {
params: {
query: search,
query: parsedSearch.query,
folder_id: selectedFolderId,
media_kind: mediaFilter === "all" ? null : mediaFilter,
favorites_only: favoritesOnly,
rating_min: minimumRating > 0 ? minimumRating : null,
limit: PAGE_SIZE,
},
});
if (requestToken !== galleryRequestToken) return;
set({
images,
totalImages: images.length,
loadedCount: images.length,
loadingImages: false,
collectionTitle: `Semantic search: ${search}`,
collectionTitle: `Semantic search: ${parsedSearch.query}`,
selectedFolderId,
similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false,
similarFolderId: null,
});
return;
}
if (parsedSearch.mode === "tag" && parsedSearch.query) {
const images = await invoke<ImageRecord[]>("search_images_by_tag", {
params: {
query: parsedSearch.query,
folder_id: selectedFolderId,
media_kind: mediaFilter === "all" ? null : mediaFilter,
favorites_only: favoritesOnly,
rating_min: minimumRating > 0 ? minimumRating : null,
limit: PAGE_SIZE,
},
});
if (requestToken !== galleryRequestToken) return;
set({
images,
totalImages: images.length,
loadedCount: images.length,
loadingImages: false,
collectionTitle: `Tag search: ${parsedSearch.query}`,
selectedFolderId,
similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false,
similarFolderId: null,
});
return;
}
@@ -543,9 +739,10 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
}>("get_images", {
params: {
folder_id: selectedFolderId,
search: search || null,
search: parsedSearch.query || null,
media_kind: mediaFilter === "all" ? null : mediaFilter,
favorites_only: favoritesOnly,
rating_min: minimumRating > 0 ? minimumRating : null,
embedding_failed_only: failedEmbeddingsOnly,
sort,
offset,
@@ -553,6 +750,7 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
},
});
if (requestToken !== galleryRequestToken) return;
set((state) => ({
images: reset ? result.images : [...state.images, ...result.images],
totalImages: result.total,
@@ -560,20 +758,24 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
loadingImages: false,
collectionTitle: reset ? null : state.collectionTitle,
similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false,
similarFolderId: null,
}));
} catch (error) {
if (requestToken !== galleryRequestToken) return;
console.error("Failed to load media:", error);
set({ loadingImages: false, imageLoadError: String(error) });
}
},
loadMoreImages: async () => {
const { loadedCount, totalImages, loadingImages, collectionTitle, similarSourceImageId, similarHasMore, selectedFolderId } = get();
const { loadedCount, totalImages, loadingImages, collectionTitle, similarSourceImageId, similarHasMore, similarFolderId } = get();
if (loadingImages || loadedCount >= totalImages) return;
if (collectionTitle === "Explore Cluster") return;
if (collectionTitle === "Similar Images" && similarSourceImageId !== null) {
if (!similarHasMore) return;
await get().loadSimilarImages(similarSourceImageId, selectedFolderId, false);
await get().loadSimilarImages(similarSourceImageId, similarFolderId, false);
return;
}
await get().loadImages(false);
@@ -614,6 +816,11 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
void get().loadImages(true);
},
setMinimumRating: (minimumRating) => {
set({ minimumRating, images: [], loadedCount: 0, collectionTitle: null, similarSourceImageId: null, similarHasMore: false, imageLoadError: null });
void get().loadImages(true);
},
setFailedEmbeddingsOnly: (failedEmbeddingsOnly) => {
set({ failedEmbeddingsOnly, images: [], loadedCount: 0, collectionTitle: null, similarSourceImageId: null, similarHasMore: false, imageLoadError: null });
void get().loadImages(true);
@@ -626,59 +833,148 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
setView: (activeView) => set({ activeView }),
setExploreMode: (exploreMode) => set({ exploreMode }),
loadTagCloud: async () => {
const { selectedFolderId, tagCloudFolderId, tagCloudLoading } = get();
// Skip if already loaded for this folder and not currently loading
if (!tagCloudLoading && tagCloudFolderId !== undefined && tagCloudFolderId === selectedFolderId) {
return;
}
const requestToken = ++tagCloudRequestToken;
set({ tagCloudLoading: true, tagCloudFolderId: selectedFolderId });
try {
const entries = await invoke<TagCloudEntry[]>("get_tag_cloud", {
folderId: selectedFolderId,
});
if (requestToken !== tagCloudRequestToken) return;
set({ tagCloudEntries: entries, tagCloudLoading: false });
} catch (error) {
if (requestToken !== tagCloudRequestToken) return;
console.error("Failed to load tag cloud:", error);
set({ tagCloudLoading: false });
}
},
searchByTag: (imageId) => {
const { selectedFolderId } = get();
set((state) => ({ activeView: "gallery", images: [], loadedCount: 0, loadingImages: true, collectionTitle: "Similar Images", imageLoadError: null, galleryScrollResetKey: state.galleryScrollResetKey + 1 }));
void get().loadSimilarImages(imageId, selectedFolderId);
loadExploreTags: async () => {
const { selectedFolderId, exploreTagsFolderId, exploreTagLoading } = get();
if (!exploreTagLoading && exploreTagsFolderId !== undefined && exploreTagsFolderId === selectedFolderId) {
return;
}
const requestToken = ++exploreTagRequestToken;
set({ exploreTagLoading: true, exploreTagsFolderId: selectedFolderId });
try {
const entries = await invoke<ExploreTagEntry[]>("get_explore_tags", {
params: { folder_id: selectedFolderId, limit: 48 },
});
if (requestToken !== exploreTagRequestToken) return;
set({ exploreTagEntries: entries, exploreTagLoading: false });
} catch (error) {
if (requestToken !== exploreTagRequestToken) return;
console.error("Failed to load explore tags:", error);
set({ exploreTagLoading: false });
}
},
loadSimilarImages: async (imageId, folderId = get().selectedFolderId, reset = true) => {
const requestedLimit = reset ? PAGE_SIZE : get().loadedCount + PAGE_SIZE;
showVisualCluster: async (imageIds) => {
const requestToken = ++similarRequestToken;
set((state) => ({
images: reset ? [] : get().images,
loadedCount: reset ? 0 : get().loadedCount,
activeView: "gallery",
search: "",
images: [],
totalImages: imageIds.length,
loadedCount: 0,
loadingImages: true,
collectionTitle: "Explore Cluster",
imageLoadError: null,
similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false,
similarFolderId: null,
galleryScrollResetKey: state.galleryScrollResetKey + 1,
}));
try {
const images = await invoke<ImageRecord[]>("get_images_by_ids", {
params: { image_ids: imageIds },
});
if (requestToken !== similarRequestToken) return;
set({
images,
totalImages: images.length,
loadedCount: images.length,
loadingImages: false,
imageLoadError: null,
collectionTitle: "Explore Cluster",
});
} catch (error) {
if (requestToken !== similarRequestToken) return;
set({
images: [],
totalImages: 0,
loadedCount: 0,
loadingImages: false,
imageLoadError: String(error),
collectionTitle: "Explore Cluster",
});
}
},
searchForTag: (tag) => {
set({ activeView: "gallery", search: `/t ${tag}`, images: [], loadedCount: 0, collectionTitle: null, similarSourceImageId: null, similarHasMore: false, similarFolderId: null, imageLoadError: null });
void get().loadImages(true);
},
loadSimilarImages: async (imageId, folderId = get().selectedFolderId, reset = true, sourceFolderId = folderId ?? null) => {
const requestToken = ++similarRequestToken;
const offset = reset ? 0 : get().loadedCount;
const similarScope = folderId === null ? "all_media" : "current_folder";
set((state) => ({
images: reset ? [] : state.images,
loadedCount: reset ? 0 : state.loadedCount,
loadingImages: true,
collectionTitle: "Similar Images",
imageLoadError: null,
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarFolderId: folderId ?? null,
similarScope,
galleryScrollResetKey: reset ? state.galleryScrollResetKey + 1 : state.galleryScrollResetKey,
}));
try {
const images = await invoke<ImageRecord[]>("find_similar_images", {
params: { image_id: imageId, folder_id: folderId ?? null, limit: requestedLimit },
const result = await invoke<SimilarImagesPage>("find_similar_images", {
params: {
image_id: imageId,
folder_id: folderId ?? null,
offset,
limit: PAGE_SIZE,
threshold: SIMILAR_DISTANCE_THRESHOLD,
},
});
const hasMore = images.length >= requestedLimit;
set({
images,
totalImages: hasMore ? images.length + PAGE_SIZE : images.length,
loadedCount: images.length,
if (requestToken !== similarRequestToken) return;
set((state) => {
const nextImages = reset ? result.images : [...state.images, ...result.images];
const nextLoadedCount = nextImages.length;
return {
images: nextImages,
totalImages: result.has_more ? nextLoadedCount + 1 : nextLoadedCount,
loadedCount: nextLoadedCount,
loadingImages: false,
imageLoadError: null,
collectionTitle: "Similar Images",
similarSourceImageId: imageId,
similarHasMore: hasMore,
selectedFolderId: folderId ?? null,
selectedImage: reset ? null : get().selectedImage,
similarSourceFolderId: sourceFolderId,
similarHasMore: result.has_more,
similarFolderId: folderId ?? null,
similarScope,
selectedImage: reset ? null : state.selectedImage,
};
});
} catch (error) {
if (requestToken !== similarRequestToken) return;
console.error("Failed to load similar images:", error);
set({
images: [],
@@ -688,13 +984,89 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
imageLoadError: String(error),
collectionTitle: "Similar Images",
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarHasMore: false,
selectedFolderId: folderId ?? null,
similarFolderId: folderId ?? null,
similarScope,
selectedImage: null,
});
}
},
loadSimilarByRegion: async (imageId, crop, folderId = get().selectedFolderId, sourceFolderId = folderId ?? null) => {
const requestToken = ++similarRequestToken;
const similarScope = folderId === null ? "all_media" : "current_folder";
set((state) => ({
images: [],
loadedCount: 0,
loadingImages: true,
collectionTitle: "Region Search Results",
imageLoadError: null,
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarFolderId: folderId ?? null,
similarScope,
galleryScrollResetKey: state.galleryScrollResetKey + 1,
selectedImage: null,
}));
try {
const result = await invoke<SimilarImagesPage>("find_similar_by_region", {
params: {
image_id: imageId,
crop_x: crop.x,
crop_y: crop.y,
crop_w: crop.w,
crop_h: crop.h,
folder_id: folderId ?? null,
offset: 0,
limit: PAGE_SIZE,
},
});
if (requestToken !== similarRequestToken) return;
set({
images: result.images,
totalImages: result.has_more ? result.images.length + 1 : result.images.length,
loadedCount: result.images.length,
loadingImages: false,
imageLoadError: null,
collectionTitle: "Region Search Results",
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarHasMore: result.has_more,
similarFolderId: folderId ?? null,
similarScope,
});
} catch (error) {
if (requestToken !== similarRequestToken) return;
console.error("Failed to load region search results:", error);
set({
images: [],
totalImages: 0,
loadedCount: 0,
loadingImages: false,
imageLoadError: String(error),
collectionTitle: "Region Search Results",
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarHasMore: false,
similarFolderId: folderId ?? null,
similarScope,
selectedImage: null,
});
}
},
setSimilarScope: (similarScope) => {
set({ similarScope });
const { similarSourceImageId, similarSourceFolderId, selectedFolderId } = get();
if (similarSourceImageId === null) return;
const folderId = similarScope === "current_folder" ? (similarSourceFolderId ?? selectedFolderId) : null;
void get().loadSimilarImages(similarSourceImageId, folderId, true, similarSourceFolderId);
},
suggestImageTags: async (imageId) => {
return invoke<string[]>("suggest_image_tags", {
params: { image_id: imageId, limit: 2 },
@@ -833,6 +1205,27 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
setSettingsOpen: (settingsOpen) => set({ settingsOpen }),
setTaggingQueueScope: (taggingQueueScope) => {
set((state) => ({
taggingQueueScope,
taggingQueueFolderIds:
taggingQueueScope === "selected" && state.taggingQueueFolderIds.length === 0 && state.folders.length > 0
? [state.folders[0].id]
: state.taggingQueueFolderIds,
}));
},
toggleTaggingQueueFolder: (folderId) => {
set((state) => {
const next = state.taggingQueueFolderIds.includes(folderId)
? state.taggingQueueFolderIds.filter((id) => id !== folderId)
: [...state.taggingQueueFolderIds, folderId].sort((a, b) => a - b);
return { taggingQueueFolderIds: next };
});
},
setTaggingQueueFolderIds: (taggingQueueFolderIds) => set({ taggingQueueFolderIds }),
loadTaggerModelStatus: async () => {
try {
const taggerModelStatus = await invoke<TaggerModelStatus>("get_tagger_model_status");
@@ -874,6 +1267,22 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
set({ taggerThreshold });
},
loadTaggerBatchSize: async () => {
try {
const taggerBatchSize = await invoke<number>("get_tagger_batch_size");
set({ taggerBatchSize });
} catch (error) {
set({ taggerModelError: String(error) });
}
},
setTaggerBatchSize: async (batchSize) => {
const taggerBatchSize = await invoke<number>("set_tagger_batch_size", {
params: { batch_size: batchSize },
});
set({ taggerBatchSize });
},
prepareTaggerModel: async () => {
set({ taggerModelPreparing: true, taggerModelError: null, taggerModelProgress: null });
try {
@@ -912,6 +1321,14 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
return queued;
},
queueTaggingJobsForFolders: async (folderIds) => {
const queued = await invoke<number>("queue_tagging_jobs", {
params: { folder_id: null, folder_ids: folderIds, image_id: null },
});
await get().loadBackgroundJobProgress();
return queued;
},
queueTaggingForImage: async (imageId) => {
const queued = await invoke<number>("queue_tagging_jobs", {
params: { folder_id: null, image_id: imageId },
@@ -928,6 +1345,14 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
return cleared;
},
clearTaggingJobsForFolders: async (folderIds) => {
const cleared = await invoke<number>("clear_tagging_jobs", {
params: { folder_id: null, folder_ids: folderIds },
});
await get().loadBackgroundJobProgress();
return cleared;
},
getImageTags: async (imageId) => {
return invoke<ImageTag[]>("get_image_tags", {
params: { image_id: imageId },
@@ -946,6 +1371,73 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
});
},
loadDuplicateScanCache: async (folderId = null) => {
interface CacheResult { groups: DuplicateGroup[]; scanned_at: number }
const cached = await invoke<CacheResult | null>("load_duplicate_scan_cache", { folderId: folderId ?? null });
if (cached) {
set({ duplicateGroups: cached.groups, duplicateLastScanned: cached.scanned_at });
}
},
scanDuplicates: async (folderId = null) => {
const { listen } = await import("@tauri-apps/api/event");
set({ duplicateScanning: true, duplicateGroups: [], duplicateScanProgress: null, duplicateSelectedIds: new Set() });
const unlisten = await listen<[number, number]>("duplicate_scan_progress", (event) => {
const [scanned, total] = event.payload;
set({ duplicateScanProgress: { scanned, total } });
});
try {
const groups = await invoke<DuplicateGroup[]>("find_duplicates", { folderId: folderId ?? null });
set({ duplicateGroups: groups, duplicateLastScanned: Math.floor(Date.now() / 1000) });
} finally {
unlisten();
set({ duplicateScanning: false });
}
},
toggleDuplicateSelected: (imageId) => {
set((state) => {
const next = new Set(state.duplicateSelectedIds);
if (next.has(imageId)) next.delete(imageId);
else next.add(imageId);
return { duplicateSelectedIds: next };
});
},
selectAllDuplicates: (imageIds) => {
set((state) => {
const next = new Set(state.duplicateSelectedIds);
for (const id of imageIds) next.add(id);
return { duplicateSelectedIds: next };
});
},
selectKeepFirstAllGroups: () => {
const { duplicateGroups } = get();
const toMark = new Set<number>();
for (const group of duplicateGroups) {
for (const img of group.images.slice(1)) toMark.add(img.id);
}
set({ duplicateSelectedIds: toMark });
},
clearDuplicateSelection: () => set({ duplicateSelectedIds: new Set() }),
deleteSelectedDuplicates: async () => {
const { duplicateSelectedIds } = get();
const ids = Array.from(duplicateSelectedIds);
if (ids.length === 0) return 0;
const deleted = await invoke<number>("delete_images_from_disk", { params: { image_ids: ids } });
// Remove deleted images from groups and drop now-trivial groups
set((state) => ({
duplicateSelectedIds: new Set(),
duplicateGroups: state.duplicateGroups
.map((g) => ({ ...g, images: g.images.filter((img) => !duplicateSelectedIds.has(img.id)) }))
.filter((g) => g.images.length > 1),
}));
return deleted;
},
retryFailedEmbeddings: async (folderId) => {
await invoke("retry_failed_embeddings", { params: { folder_id: folderId } });
await get().loadBackgroundJobProgress();
@@ -979,7 +1471,9 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
if (progress.done) {
void get().loadFolders();
void get().loadBackgroundJobProgress();
if (get().activeView !== "explore" && !isDerivedCollectionTitle(get().collectionTitle)) {
void get().loadImages(true);
}
setTimeout(() => {
set((state) => {
@@ -1019,12 +1513,17 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
const batch = event.payload;
set((state) => {
if (isDerivedCollectionTitle(state.collectionTitle) || state.activeView === "explore") {
return state;
}
const visibleImages = batch.images.filter((image) =>
matchesFilters(
image,
state.selectedFolderId,
state.mediaFilter,
state.favoritesOnly,
state.minimumRating,
state.failedEmbeddingsOnly,
state.search,
),
@@ -1049,12 +1548,21 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
const batch = event.payload;
set((state) => {
if (isDerivedCollectionTitle(state.collectionTitle) || state.activeView === "explore") {
const selectedImage =
state.selectedImage && batch.images.some((image) => image.id === state.selectedImage?.id)
? batch.images.find((image) => image.id === state.selectedImage?.id) ?? state.selectedImage
: state.selectedImage;
return { selectedImage };
}
const visibleImages = batch.images.filter((image) =>
matchesFilters(
image,
state.selectedFolderId,
state.mediaFilter,
state.favoritesOnly,
state.minimumRating,
state.failedEmbeddingsOnly,
state.search,
),