Compare commits

...

5 Commits

Author SHA1 Message Date
LyAhn d7595703de docs: rewrite README to reflect current features, add CLAUDE.md 2026-06-02 19:40:00 +01:00
LyAhn 5b35bc5b6e chore: set app title to Phokus, remove default Vite favicon 2026-06-02 19:39:44 +01:00
LyAhn df17497808 feat: add region-based similarity search
Adds a "Search within image" button to the lightbox that lets the user
draw a crop region on an image and find visually similar results using
that crop's embedding. The crop is embedded in-memory without a temp
file via a new embed_image_crop method on ClipImageEmbedder.

Introduces a folder-scoped cosine search (search_image_ids_by_embedding_in_folder)
to support the current-folder scope option, and wires up the new
find_similar_by_region Tauri command end-to-end from Rust through to
the Zustand store and Lightbox UI.
2026-06-02 19:39:31 +01:00
LyAhn ff4a568b57 feat: expand media exploration and tagging controls 2026-04-12 12:18:47 +01:00
LyAhn b2826d1143 feat: add WD tagger with CSV tag support and model download via ureq/zip
Introduces tagger.rs for WD-based image tagging with CSV label support.
Adds csv, ureq, and zip dependencies; updates gitignore to exclude Python/JSON files.
2026-04-08 20:04:37 +01:00
29 changed files with 5935 additions and 1014 deletions
+6
View File
@@ -29,3 +29,9 @@ dist-ssr
# Local staging area # Local staging area
/staging /staging
# Misc
*.py
*.json
*.pyc
+94
View File
@@ -0,0 +1,94 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project overview
**Phokus** — a Tauri v2 desktop image gallery app with a React/TypeScript frontend and a Rust backend. The app indexes local media folders, generates thumbnails via FFmpeg, computes visual embeddings for semantic search and similarity, and AI-tags images using the WD tagger (ONNX via `ort`). AI captioning code exists in the backend but the UI surface has been removed; the worker is commented out in `lib.rs`.
## Commands
```bash
# Development (Vite hot-reload + Rust auto-rebuild)
pnpm dev:app
# Frontend only (no Tauri window)
pnpm dev:vite
# Production build
pnpm build:app
# Type-check frontend
pnpm build:vite
```
Use **pnpm** — never npm.
There are no test suites configured.
## Architecture
### Frontend (`src/`)
- **`store.ts`** — single Zustand store (`useGalleryStore`) that owns all app state and all `invoke()` calls to the Tauri backend. Every feature (folders, images, search, similar images, tags, captions, tagger, duplicates) is implemented as store actions here. React components are thin consumers.
- **`App.tsx`** — sets up Tauri event listeners (`subscribeToProgress`) and renders the top-level layout (sidebar + active view).
- **`src/components/`** — UI components: `Gallery`, `Lightbox`, `Sidebar`, `Toolbar`, `TagCloud`, `DuplicateFinder`, `BackgroundTasks`, `SettingsModal`, `MenuBar`, `TitleBar`.
- State management: Zustand v5, no selectors library — components call `useGalleryStore(s => s.field)` directly.
- Styling: Tailwind CSS v4 (Vite plugin, no config file).
- Virtualized gallery grid: `@tanstack/react-virtual`.
- Animation: `framer-motion`.
### Search modes
The search bar supports prefix syntax parsed by `parseSearchValue` in `store.ts`:
- No prefix / `f:` — filename search (paginated, DB-backed)
- `/s <query>` or `s: <query>` — semantic (embedding) search
- `/t <tag>` or `t: <tag>` — tag search
### Backend (`src-tauri/src/`)
Workers are started in `lib.rs` and run as background threads throughout the app lifetime:
- **thumbnail worker** (multiple threads, count from `StorageProfile::Balanced`)
- **metadata worker** — FFmpeg probe for video files
- **embedding worker** — generates CLIP-style visual embeddings (candle, HuggingFace hub)
- **tagging worker** — WD tagger via ONNX Runtime (`ort`), DirectML/CPU acceleration
Key modules:
| File | Purpose |
|------|---------|
| `db.rs` | SQLite pool (r2d2 + rusqlite), schema migrations, all query functions |
| `commands.rs` | All `#[tauri::command]` handlers — one-to-one with frontend `invoke()` calls |
| `indexer.rs` | Worker thread launchers and job dispatch |
| `embedder.rs` | Visual embedding generation (candle + HF hub models) |
| `vector.rs` | sqlite-vec integration + HNSW index for ANN search |
| `hnsw_index.rs` | In-memory HNSW index wrapper (hnsw_rs) |
| `tagger.rs` | WD tagger: ONNX model download, inference, CSV tag loading |
| `captioner.rs` | AI captioning (ONNX, disabled in workers but code intact) |
| `thumbnail.rs` | Thumbnail generation (image crate + fast_image_resize, FFmpeg for video) |
| `media.rs` | FFmpeg sidecar provisioning and probing |
| `storage.rs` | `StorageProfile` for tuning worker counts |
Database: SQLite with WAL mode, stored in the Tauri app data directory as `gallery.db`. Thumbnails stored alongside as `thumbnails/`.
### Tauri events (backend → frontend)
| Event | Payload |
|-------|---------|
| `index-progress` | `IndexProgress` |
| `media-job-progress` | `MediaJobProgressEvent` |
| `indexed-images` | `IndexedImagesBatch` |
| `media-updated` | `ThumbnailBatch` |
| `caption-model-progress` | `CaptionModelProgress` |
| `tagger-model-progress` | `TaggerModelProgress` |
| `duplicate_scan_progress` | `[scanned, total]` |
### Key types
`ImageRecord` (mirrored in `store.ts` and `db.rs`) is the central data type. It carries embedding status, tagging status, caption data, and media metadata. The frontend type must stay in sync with the Rust struct serialization.
## Development notes
- Hot Reload is active during `dev:app` — do not restart the server for frontend changes. Restart only when adding Rust crates or changing Vite config.
- ML inference crates (`candle-*`, `ort`, `image`, `rayon`, `tokenizers`, `xxhash-rust`, `rusqlite`) use `opt-level = 3` in dev profile to keep inference performance acceptable.
- The caption worker is intentionally disabled (`lib.rs:73`) — the backend code is intact for future re-enabling.
- **Never use `any` type** in TypeScript — look up correct types.
+34 -92
View File
@@ -1,116 +1,58 @@
# Phokus # Phokus
## Overview A local-first desktop media library for browsing, filtering, and curating image and video folders.
Phokus is a Tauri desktop app for building a fast, local media library from folders on disk. It indexes images and videos, stores metadata in SQLite, and gives you a dense browsing workflow with filtering, favorites, ratings, and a lightbox preview. ## Features
The current app is optimized for: - Add and remove media folders; background indexing with live progress
- Browse all media or filter by folder, type (image/video), favorites, or star rating
- local folders instead of cloud import flows - **Filename search**, **semantic search** (`/s query`), and **tag search** (`/t tag`)
- large visual libraries - **Similar image search** — find visually similar media by image or selected region
- quick review and curation - **AI tagging** via WD tagger (ONNX, CPU/DirectML) with confidence threshold control
- mixed image and video browsing - **Explore view** — visual cluster map and tag cloud for browsing by theme
- **Duplicate finder** — scan for exact duplicates by file hash with bulk delete
## Current features - Lightbox preview with keyboard navigation, inline tag editing, and rating controls
- Sort by date, name, size, rating, or duration
- Add and remove media folders - Grid density controls (compact / comfortable / detail)
- Background indexing with progress updates
- Browse all media or filter by folder
- Search by filename
- Filter by images, videos, or favorites
- Sort by modified date, name, or file size
- Grid density controls
- Lightbox preview with keyboard navigation
- Favorite and star-rating metadata saved in SQLite
- Virtualized/local-first architecture built on Tauri + React
## Supported formats ## Supported formats
Images: | Images | Videos |
|--------|--------|
- `jpg` | jpg, jpeg, png, gif, bmp | mp4, mov, m4v |
- `jpeg` | tiff, tif, webp, avif, heic, heif | webm |
- `png`
- `gif`
- `bmp`
- `tiff`
- `tif`
- `webp`
- `avif`
- `heic`
- `heif`
Videos:
- `mp4`
- `mov`
- `m4v`
- `webm`
## Stack ## Stack
- Tauri 2 - Tauri 2 + Rust backend
- React 19 - React 19 + TypeScript + Zustand
- TypeScript - SQLite + `sqlite-vec` (vector search)
- Zustand - ONNX Runtime (`ort`) for AI tagging
- Rust - Candle (Rust ML) for visual embeddings
- SQLite + `sqlite-vec` - FFmpeg sidecar for video thumbnails and metadata
- Vite - Vite + Tailwind CSS v4
## Project structure
- `src/`: React UI, state, and components
- `src-tauri/src/commands.rs`: Tauri command surface
- `src-tauri/src/db.rs`: SQLite schema and queries
- `src-tauri/src/indexer.rs`: folder crawling and batch indexing
- `src-tauri/src/vector.rs`: vector table setup for future semantic workflows
## Development ## Development
### Prerequisites **Prerequisites:** Node.js 20+, pnpm, Rust toolchain, Tauri system prerequisites for Windows.
- Node.js 20+
- `pnpm`
- Rust toolchain
- Tauri system prerequisites for Windows
### Install
```bash ```bash
pnpm install pnpm install
```
### Run in development # Run with hot-reload (frontend + Rust)
pnpm dev:app
```bash # Frontend only
pnpm tauri dev pnpm dev:vite
```
### Build # Production build
pnpm build:app
```bash
pnpm tauri build
``` ```
## How it works ## How it works
1. Add a folder from the sidebar or Library menu. 1. Add a folder from the sidebar — the Rust indexer walks it recursively.
2. The Rust indexer walks the directory recursively. 2. Supported files are written to SQLite with metadata (path, dimensions, media type, etc.).
3. Supported files are written into SQLite with metadata such as path, size, dimensions, media type, rating, and favorite state. 3. Background workers generate thumbnails, compute visual embeddings, and run AI tagging.
4. Progress events stream back to the UI while the gallery updates incrementally. 4. Progress events stream back to the UI while the gallery updates incrementally.
5. The gallery view loads media in pages and opens items in a lightbox for review. 5. Embeddings power semantic search and the similar images feature via an HNSW index.
## Notes
- This is currently a local desktop library, not a sync product.
- Search is filename-based right now.
- The vector table and embedding fields exist, but semantic search is not wired into the UI yet.
- Some visible UI copy may still use the old working name until the frontend text is updated.
## Positioning
The clearest product description today is:
> A local-first desktop media library for browsing, filtering, and curating image and video folders.
That description is more accurate than "gallery" alone and gives you a better base for future branding, onboarding copy, and a landing page.
+1 -3
View File
@@ -2,11 +2,9 @@
<html lang="en"> <html lang="en">
<head> <head>
<meta charset="UTF-8" /> <meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Tauri + React + Typescript</title> <title>Phokus</title>
</head> </head>
<body> <body>
<div id="root"></div> <div id="root"></div>
<script type="module" src="/src/main.tsx"></script> <script type="module" src="/src/main.tsx"></script>
+7 -3
View File
@@ -4,9 +4,11 @@
"version": "0.1.0", "version": "0.1.0",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite", "build:app": "tauri build",
"build": "tsc && vite build", "build:vite": "tsc && vite build",
"preview": "vite preview", "dev:app": "tauri dev",
"dev:vite": "vite",
"preview": "vite preview",
"tauri": "tauri" "tauri": "tauri"
}, },
"dependencies": { "dependencies": {
@@ -15,6 +17,7 @@
"@tauri-apps/plugin-dialog": "^2.7.0", "@tauri-apps/plugin-dialog": "^2.7.0",
"@tauri-apps/plugin-fs": "^2.5.0", "@tauri-apps/plugin-fs": "^2.5.0",
"@tauri-apps/plugin-opener": "^2", "@tauri-apps/plugin-opener": "^2",
"d3-force": "^3.0.0",
"framer-motion": "^12.38.0", "framer-motion": "^12.38.0",
"react": "^19.1.0", "react": "^19.1.0",
"react-dom": "^19.1.0", "react-dom": "^19.1.0",
@@ -23,6 +26,7 @@
"devDependencies": { "devDependencies": {
"@tailwindcss/vite": "^4.2.2", "@tailwindcss/vite": "^4.2.2",
"@tauri-apps/cli": "^2", "@tauri-apps/cli": "^2",
"@types/d3-force": "^3.0.10",
"@types/react": "^19.1.8", "@types/react": "^19.1.8",
"@types/react-dom": "^19.1.6", "@types/react-dom": "^19.1.6",
"@vitejs/plugin-react": "^4.6.0", "@vitejs/plugin-react": "^4.6.0",
+39
View File
@@ -23,6 +23,9 @@ importers:
'@tauri-apps/plugin-opener': '@tauri-apps/plugin-opener':
specifier: ^2 specifier: ^2
version: 2.5.3 version: 2.5.3
d3-force:
specifier: ^3.0.0
version: 3.0.0
framer-motion: framer-motion:
specifier: ^12.38.0 specifier: ^12.38.0
version: 12.38.0(react-dom@19.2.4(react@19.2.4))(react@19.2.4) version: 12.38.0(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
@@ -42,6 +45,9 @@ importers:
'@tauri-apps/cli': '@tauri-apps/cli':
specifier: ^2 specifier: ^2
version: 2.10.1 version: 2.10.1
'@types/d3-force':
specifier: ^3.0.10
version: 3.0.10
'@types/react': '@types/react':
specifier: ^19.1.8 specifier: ^19.1.8
version: 19.2.14 version: 19.2.14
@@ -662,6 +668,9 @@ packages:
'@types/babel__traverse@7.28.0': '@types/babel__traverse@7.28.0':
resolution: {integrity: sha512-8PvcXf70gTDZBgt9ptxJ8elBeBjcLOAcOtoO/mPJjtji1+CdGbHgm77om1GrsPxsiE+uXIpNSK64UYaIwQXd4Q==} resolution: {integrity: sha512-8PvcXf70gTDZBgt9ptxJ8elBeBjcLOAcOtoO/mPJjtji1+CdGbHgm77om1GrsPxsiE+uXIpNSK64UYaIwQXd4Q==}
'@types/d3-force@3.0.10':
resolution: {integrity: sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw==}
'@types/estree@1.0.8': '@types/estree@1.0.8':
resolution: {integrity: sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==} resolution: {integrity: sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==}
@@ -698,6 +707,22 @@ packages:
csstype@3.2.3: csstype@3.2.3:
resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==} resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==}
d3-dispatch@3.0.1:
resolution: {integrity: sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg==}
engines: {node: '>=12'}
d3-force@3.0.0:
resolution: {integrity: sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg==}
engines: {node: '>=12'}
d3-quadtree@3.0.1:
resolution: {integrity: sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw==}
engines: {node: '>=12'}
d3-timer@3.0.1:
resolution: {integrity: sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==}
engines: {node: '>=12'}
debug@4.4.3: debug@4.4.3:
resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==} resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==}
engines: {node: '>=6.0'} engines: {node: '>=6.0'}
@@ -1448,6 +1473,8 @@ snapshots:
dependencies: dependencies:
'@babel/types': 7.29.0 '@babel/types': 7.29.0
'@types/d3-force@3.0.10': {}
'@types/estree@1.0.8': {} '@types/estree@1.0.8': {}
'@types/react-dom@19.2.3(@types/react@19.2.14)': '@types/react-dom@19.2.3(@types/react@19.2.14)':
@@ -1486,6 +1513,18 @@ snapshots:
csstype@3.2.3: {} csstype@3.2.3: {}
d3-dispatch@3.0.1: {}
d3-force@3.0.0:
dependencies:
d3-dispatch: 3.0.1
d3-quadtree: 3.0.1
d3-timer: 3.0.1
d3-quadtree@3.0.1: {}
d3-timer@3.0.1: {}
debug@4.4.3: debug@4.4.3:
dependencies: dependencies:
ms: 2.1.3 ms: 2.1.3
+12
View File
@@ -0,0 +1,12 @@
[target.x86_64-pc-windows-msvc]
rustflags = [
# Disable MSVC incremental linking — avoids .ilk file corruption
# and removes one source of link failure on restart.
"-C", "link-arg=/INCREMENTAL:NO",
# Skip PDB generation entirely in dev builds.
# The .pdb file is the most common reason the linker fails after
# an unclean shutdown: the previous phokus.exe process holds the
# file open, so link.exe cannot write a new one → LNK error.
# Rust backtraces still work without MSVC PDBs.
"-C", "link-arg=/DEBUG:NONE",
]
+332
View File
@@ -61,6 +61,73 @@ dependencies = [
"libc", "libc",
] ]
[[package]]
name = "anndists"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8396b473aa0bceed68fb32462505387ea39fa47c7029417e0a49f10592b036"
dependencies = [
"anyhow",
"cfg-if",
"cpu-time",
"env_logger",
"lazy_static",
"log",
"num-traits",
"num_cpus",
"rayon",
]
[[package]]
name = "anstream"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
dependencies = [
"anstyle",
"anstyle-parse",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
"is_terminal_polyfill",
"utf8parse",
]
[[package]]
name = "anstyle"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anstyle-parse"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
dependencies = [
"utf8parse",
]
[[package]]
name = "anstyle-query"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "anstyle-wincon"
version = "3.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys 0.61.2",
]
[[package]] [[package]]
name = "anyhow" name = "anyhow"
version = "1.0.102" version = "1.0.102"
@@ -266,6 +333,15 @@ version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "bincode"
version = "1.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad"
dependencies = [
"serde",
]
[[package]] [[package]]
name = "bit-set" name = "bit-set"
version = "0.8.0" version = "0.8.0"
@@ -595,6 +671,12 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "cfg_aliases"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]] [[package]]
name = "chacha20" name = "chacha20"
version = "0.10.0" version = "0.10.0"
@@ -626,6 +708,12 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b" checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b"
[[package]]
name = "colorchoice"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]] [[package]]
name = "combine" name = "combine"
version = "4.6.7" version = "4.6.7"
@@ -757,6 +845,16 @@ dependencies = [
"libc", "libc",
] ]
[[package]]
name = "cpu-time"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9e393a7668fe1fad3075085b86c781883000b4ede868f43627b34a87c8b7ded"
dependencies = [
"libc",
"winapi",
]
[[package]] [[package]]
name = "cpufeatures" name = "cpufeatures"
version = "0.2.17" version = "0.2.17"
@@ -874,6 +972,27 @@ dependencies = [
"syn 2.0.117", "syn 2.0.117",
] ]
[[package]]
name = "csv"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52cd9d68cf7efc6ddfaaee42e7288d3a99d613d4b50f76ce9827ae0c6e14f938"
dependencies = [
"csv-core",
"itoa",
"ryu",
"serde_core",
]
[[package]]
name = "csv-core"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "704a3c26996a80471189265814dbc2c257598b96b8a7feae2d31ace646bb9782"
dependencies = [
"memchr",
]
[[package]] [[package]]
name = "ctor" name = "ctor"
version = "0.2.9" version = "0.2.9"
@@ -1331,12 +1450,35 @@ dependencies = [
"syn 2.0.117", "syn 2.0.117",
] ]
[[package]]
name = "env_filter"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32e90c2accc4b07a8456ea0debdc2e7587bdd890680d71173a15d4ae604f6eef"
dependencies = [
"log",
"regex",
]
[[package]] [[package]]
name = "env_home" name = "env_home"
version = "0.1.0" version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7f84e12ccf0a7ddc17a6c41c93326024c42920d7ee630d04950e6926645c0fe" checksum = "c7f84e12ccf0a7ddc17a6c41c93326024c42920d7ee630d04950e6926645c0fe"
[[package]]
name = "env_logger"
version = "0.11.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0621c04f2196ac3f488dd583365b9c09be011a4ab8b9f37248ffcc8f6198b56a"
dependencies = [
"anstream",
"anstyle",
"env_filter",
"jiff",
"log",
]
[[package]] [[package]]
name = "equivalent" name = "equivalent"
version = "1.0.2" version = "1.0.2"
@@ -2319,6 +2461,8 @@ version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1" checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [ dependencies = [
"allocator-api2",
"equivalent",
"foldhash 0.1.5", "foldhash 0.1.5",
] ]
@@ -2395,6 +2539,31 @@ version = "1.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec9d92d097f4749b64e8cc33d924d9f40a2d4eb91402b458014b781f5733d60f" checksum = "ec9d92d097f4749b64e8cc33d924d9f40a2d4eb91402b458014b781f5733d60f"
[[package]]
name = "hnsw_rs"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43a5258f079b97bf2e8311ff9579e903c899dcbac0d9a138d62e9a066778bd07"
dependencies = [
"anndists",
"anyhow",
"bincode",
"cfg-if",
"cpu-time",
"env_logger",
"hashbrown 0.15.5",
"indexmap 2.13.1",
"lazy_static",
"log",
"mmap-rs",
"num-traits",
"num_cpus",
"parking_lot",
"rand 0.9.2",
"rayon",
"serde",
]
[[package]] [[package]]
name = "html5ever" name = "html5ever"
version = "0.29.1" version = "0.29.1"
@@ -2798,6 +2967,12 @@ dependencies = [
"once_cell", "once_cell",
] ]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
[[package]] [[package]]
name = "itertools" name = "itertools"
version = "0.14.0" version = "0.14.0"
@@ -2836,6 +3011,30 @@ dependencies = [
"system-deps", "system-deps",
] ]
[[package]]
name = "jiff"
version = "0.2.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a3546dc96b6d42c5f24902af9e2538e82e39ad350b0c766eb3fbf2d8f3d8359"
dependencies = [
"jiff-static",
"log",
"portable-atomic",
"portable-atomic-util",
"serde_core",
]
[[package]]
name = "jiff-static"
version = "0.2.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a8c8b344124222efd714b73bb41f8b5120b27a7cc1c75593a6ff768d9d05aa4"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]] [[package]]
name = "jni" name = "jni"
version = "0.21.1" version = "0.21.1"
@@ -2937,6 +3136,12 @@ dependencies = [
"selectors 0.24.0", "selectors 0.24.0",
] ]
[[package]]
name = "lazy_static"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]] [[package]]
name = "leb128fmt" name = "leb128fmt"
version = "0.1.0" version = "0.1.0"
@@ -3088,6 +3293,15 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4" checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
[[package]]
name = "mach2"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d640282b302c0bb0a2a8e0233ead9035e3bed871f0b7e81fe4a1ec829765db44"
dependencies = [
"libc",
]
[[package]] [[package]]
name = "macro_rules_attribute" name = "macro_rules_attribute"
version = "0.2.2" version = "0.2.2"
@@ -3214,6 +3428,23 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "mmap-rs"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ecce9d566cb9234ae3db9e249c8b55665feaaf32b0859ff1e27e310d2beb3d8"
dependencies = [
"bitflags 2.11.0",
"combine",
"libc",
"mach2",
"nix",
"sysctl",
"thiserror 2.0.18",
"widestring",
"windows 0.48.0",
]
[[package]] [[package]]
name = "monostate" name = "monostate"
version = "0.1.18" version = "0.1.18"
@@ -3335,6 +3566,18 @@ version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086" checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
[[package]]
name = "nix"
version = "0.30.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6"
dependencies = [
"bitflags 2.11.0",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]] [[package]]
name = "nodrop" name = "nodrop"
version = "0.1.14" version = "0.1.14"
@@ -3613,6 +3856,12 @@ version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "once_cell_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]] [[package]]
name = "onig" name = "onig"
version = "6.5.1" version = "6.5.1"
@@ -4009,11 +4258,14 @@ dependencies = [
"candle-nn", "candle-nn",
"candle-transformers", "candle-transformers",
"chrono", "chrono",
"csv",
"fast_image_resize", "fast_image_resize",
"ffmpeg-sidecar", "ffmpeg-sidecar",
"hf-hub", "hf-hub",
"hnsw_rs",
"image", "image",
"log", "log",
"memmap2",
"ort", "ort",
"r2d2", "r2d2",
"r2d2_sqlite", "r2d2_sqlite",
@@ -4030,9 +4282,11 @@ dependencies = [
"tauri-plugin-opener", "tauri-plugin-opener",
"tokenizers", "tokenizers",
"tokio", "tokio",
"ureq",
"uuid", "uuid",
"walkdir", "walkdir",
"xxhash-rust", "xxhash-rust",
"zip 4.6.1",
] ]
[[package]] [[package]]
@@ -6551,6 +6805,12 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]]
name = "utf8parse"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]] [[package]]
name = "uuid" name = "uuid"
version = "1.23.0" version = "1.23.0"
@@ -6914,6 +7174,12 @@ dependencies = [
"winsafe", "winsafe",
] ]
[[package]]
name = "widestring"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72069c3113ab32ab29e5584db3c6ec55d416895e60715417b5b883a357c3e471"
[[package]] [[package]]
name = "winapi" name = "winapi"
version = "0.3.9" version = "0.3.9"
@@ -6960,6 +7226,15 @@ dependencies = [
"windows-version", "windows-version",
] ]
[[package]]
name = "windows"
version = "0.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e686886bc078bc1b0b600cac0147aadb815089b6e4da64016cbd754b6342700f"
dependencies = [
"windows-targets 0.48.5",
]
[[package]] [[package]]
name = "windows" name = "windows"
version = "0.61.3" version = "0.61.3"
@@ -7212,6 +7487,21 @@ dependencies = [
"windows_x86_64_msvc 0.42.2", "windows_x86_64_msvc 0.42.2",
] ]
[[package]]
name = "windows-targets"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c"
dependencies = [
"windows_aarch64_gnullvm 0.48.5",
"windows_aarch64_msvc 0.48.5",
"windows_i686_gnu 0.48.5",
"windows_i686_msvc 0.48.5",
"windows_x86_64_gnu 0.48.5",
"windows_x86_64_gnullvm 0.48.5",
"windows_x86_64_msvc 0.48.5",
]
[[package]] [[package]]
name = "windows-targets" name = "windows-targets"
version = "0.52.6" version = "0.52.6"
@@ -7278,6 +7568,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8" checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8"
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8"
[[package]] [[package]]
name = "windows_aarch64_gnullvm" name = "windows_aarch64_gnullvm"
version = "0.52.6" version = "0.52.6"
@@ -7296,6 +7592,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43" checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43"
[[package]]
name = "windows_aarch64_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc"
[[package]] [[package]]
name = "windows_aarch64_msvc" name = "windows_aarch64_msvc"
version = "0.52.6" version = "0.52.6"
@@ -7314,6 +7616,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f" checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f"
[[package]]
name = "windows_i686_gnu"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e"
[[package]] [[package]]
name = "windows_i686_gnu" name = "windows_i686_gnu"
version = "0.52.6" version = "0.52.6"
@@ -7344,6 +7652,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060" checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060"
[[package]]
name = "windows_i686_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406"
[[package]] [[package]]
name = "windows_i686_msvc" name = "windows_i686_msvc"
version = "0.52.6" version = "0.52.6"
@@ -7362,6 +7676,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36" checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36"
[[package]]
name = "windows_x86_64_gnu"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e"
[[package]] [[package]]
name = "windows_x86_64_gnu" name = "windows_x86_64_gnu"
version = "0.52.6" version = "0.52.6"
@@ -7380,6 +7700,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3" checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc"
[[package]] [[package]]
name = "windows_x86_64_gnullvm" name = "windows_x86_64_gnullvm"
version = "0.52.6" version = "0.52.6"
@@ -7398,6 +7724,12 @@ version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0" checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0"
[[package]]
name = "windows_x86_64_msvc"
version = "0.48.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538"
[[package]] [[package]]
name = "windows_x86_64_msvc" name = "windows_x86_64_msvc"
version = "0.52.6" version = "0.52.6"
+54 -1
View File
@@ -27,6 +27,7 @@ rusqlite = { version = "0.32", features = ["bundled"] }
r2d2 = "0.8" r2d2 = "0.8"
r2d2_sqlite = "0.25" r2d2_sqlite = "0.25"
sqlite-vec = "=0.1.9" sqlite-vec = "=0.1.9"
hnsw_rs = "0.3.4"
image = { version = "0.25", default-features = false, features = ["jpeg", "png", "webp", "gif", "bmp", "tiff"] } image = { version = "0.25", default-features = false, features = ["jpeg", "png", "webp", "gif", "bmp", "tiff"] }
fast_image_resize = { version = "6.0.0", features = ["image"] } fast_image_resize = { version = "6.0.0", features = ["image"] }
walkdir = "2" walkdir = "2"
@@ -38,10 +39,62 @@ anyhow = "1"
log = "0.4" log = "0.4"
ffmpeg-sidecar = "2.5.0" ffmpeg-sidecar = "2.5.0"
xxhash-rust = { version = "0.8", features = ["xxh3"] } xxhash-rust = { version = "0.8", features = ["xxh3"] }
memmap2 = "0.9"
sysinfo = "0.38.4" sysinfo = "0.38.4"
candle-core = { version = "0.10.2", features = ["cuda"] } candle-core = { version = "0.10.2", features = ["cuda"] }
candle-nn = { version = "0.10.2", features = ["cuda"] } candle-nn = { version = "0.10.2", features = ["cuda"] }
candle-transformers = { version = "0.10.2", features = ["cuda"] } candle-transformers = { version = "0.10.2", features = ["cuda"] }
hf-hub = { version = "0.5.0", default-features = false, features = ["ureq", "native-tls"] } hf-hub = { version = "0.5.0", default-features = false, features = ["ureq", "native-tls"] }
tokenizers = "0.22.1" tokenizers = "0.22.1"
ort = { version = "2.0.0-rc.12", default-features = false, features = ["std", "ndarray", "download-binaries", "copy-dylibs", "load-dynamic", "api-24", "tls-native"] } ort = { version = "2.0.0-rc.12", default-features = false, features = ["std", "ndarray", "download-binaries", "copy-dylibs", "load-dynamic", "directml", "api-24", "tls-native"] }
ureq = { version = "3.3.0", default-features = false, features = ["native-tls"] }
zip = { version = "4.6.1", default-features = false, features = ["deflate"] }
csv = "1"
# ── Dev-mode performance ────────────────────────────────────────────────────
# opt-level=1 on the main crate keeps incremental compile short.
# Only the packages that are genuine hot-path bottlenecks in dev get opt-level=3
# (using "*" caused cargo to recheck all dependency fingerprints too aggressively,
# which caused spurious full rebuilds and linker conflicts).
[profile.dev]
opt-level = 1
# ML inference — without opt these run 20-50× slower than release
[profile.dev.package.candle-core]
opt-level = 3
[profile.dev.package.candle-nn]
opt-level = 3
[profile.dev.package.candle-transformers]
opt-level = 3
# ONNX runtime (WD tagger)
[profile.dev.package.ort]
opt-level = 3
[profile.dev.package.ort-sys]
opt-level = 3
# Image decode/resize workers
[profile.dev.package.image]
opt-level = 3
[profile.dev.package.fast_image_resize]
opt-level = 3
# Parallel work scheduler
[profile.dev.package.rayon]
opt-level = 3
[profile.dev.package.rayon-core]
opt-level = 3
# Tokenisation (embedding model pre-processing)
[profile.dev.package.tokenizers]
opt-level = 3
# Hashing (duplicate finder)
[profile.dev.package.xxhash-rust]
opt-level = 3
# SQLite (frequent db calls in workers)
[profile.dev.package.rusqlite]
opt-level = 3
[profile.dev.package.libsqlite3-sys]
opt-level = 3
+496 -106
View File
@@ -1,18 +1,54 @@
use anyhow::Result; use anyhow::Result;
use hf_hub::{api::sync::Api, Repo, RepoType}; use hf_hub::{api::sync::Api, Repo, RepoType};
use image::{imageops::FilterType, ImageReader}; use image::{imageops::FilterType, ImageReader};
use ort::ep;
use ort::session::SessionInputValue; use ort::session::SessionInputValue;
use ort::session::SessionOutputs;
use ort::session::{builder::GraphOptimizationLevel, Session}; use ort::session::{builder::GraphOptimizationLevel, Session};
use ort::value::{Shape, Tensor}; use ort::value::{Shape, Tensor};
use serde::Serialize; use serde::{Deserialize, Serialize};
use std::borrow::Cow; use std::borrow::Cow;
use std::io::{Cursor, Read};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::OnceLock;
use std::time::Instant;
use tokenizers::Tokenizer; use tokenizers::Tokenizer;
pub const FLORENCE_MODEL_ID: &str = "onnx-community/Florence-2-base-ft"; pub const FLORENCE_MODEL_ID: &str = "onnx-community/Florence-2-base-ft";
pub const FLORENCE_CAPTION_MODEL_NAME: &str = "florence-2-base-ft-onnx-q4"; pub const FLORENCE_CAPTION_MODEL_NAME: &str = "florence-2-base-ft-onnx-q4";
const ONNX_RUNTIME_NUGET_URL: &str =
"https://www.nuget.org/api/v2/package/Microsoft.ML.OnnxRuntime.DirectML/1.24.2";
const DIRECTML_NUGET_URL: &str =
"https://www.nuget.org/api/v2/package/Microsoft.AI.DirectML/1.15.4";
const ONNX_RUNTIME_DLL_FILE: &str = "onnxruntime/onnxruntime.dll";
const ONNX_RUNTIME_PROVIDERS_DLL_FILE: &str = "onnxruntime/onnxruntime_providers_shared.dll";
const DIRECTML_DLL_FILE: &str = "onnxruntime/DirectML.dll";
const CAPTION_ACCELERATION_FILE: &str = "settings/caption_acceleration.txt";
const CAPTION_DETAIL_FILE: &str = "settings/caption_detail.txt";
const ONNX_RUNTIME_FILES: &[(&str, &str, &str)] = &[
(
ONNX_RUNTIME_DLL_FILE,
ONNX_RUNTIME_NUGET_URL,
"runtimes/win-x64/native/onnxruntime.dll",
),
(
ONNX_RUNTIME_PROVIDERS_DLL_FILE,
ONNX_RUNTIME_NUGET_URL,
"runtimes/win-x64/native/onnxruntime_providers_shared.dll",
),
(
DIRECTML_DLL_FILE,
DIRECTML_NUGET_URL,
"bin/x64-win/DirectML.dll",
),
];
const REQUIRED_FILES: &[&str] = &[ const REQUIRED_FILES: &[&str] = &[
ONNX_RUNTIME_DLL_FILE,
ONNX_RUNTIME_PROVIDERS_DLL_FILE,
DIRECTML_DLL_FILE,
"config.json", "config.json",
"generation_config.json", "generation_config.json",
"preprocessor_config.json", "preprocessor_config.json",
@@ -21,10 +57,81 @@ const REQUIRED_FILES: &[&str] = &[
"special_tokens_map.json", "special_tokens_map.json",
"onnx/vision_encoder_fp16.onnx", "onnx/vision_encoder_fp16.onnx",
"onnx/encoder_model_q4.onnx", "onnx/encoder_model_q4.onnx",
"onnx/decoder_model_q4.onnx",
"onnx/decoder_model_merged_q4.onnx", "onnx/decoder_model_merged_q4.onnx",
"onnx/embed_tokens_fp16.onnx", "onnx/embed_tokens_fp16.onnx",
]; ];
static ORT_RUNTIME_INIT: OnceLock<std::result::Result<(), String>> = OnceLock::new();
/// Set to `true` by `set_caption_acceleration` so the caption worker loop
/// knows to drop its cached `FlorenceCaptioner` and reload with the new EP.
pub static CAPTION_SESSION_DIRTY: AtomicBool = AtomicBool::new(false);
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum CaptionAcceleration {
Auto,
Cpu,
Directml,
}
impl CaptionAcceleration {
fn as_str(self) -> &'static str {
match self {
Self::Auto => "auto",
Self::Cpu => "cpu",
Self::Directml => "directml",
}
}
}
impl Default for CaptionAcceleration {
fn default() -> Self {
Self::Auto
}
}
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum CaptionDetail {
Short,
Detailed,
Paragraph,
}
impl CaptionDetail {
fn as_str(self) -> &'static str {
match self {
Self::Short => "short",
Self::Detailed => "detailed",
Self::Paragraph => "paragraph",
}
}
fn prompt(self) -> &'static str {
match self {
Self::Short => "What does the image describe?",
Self::Detailed => "Describe in detail what is shown in the image.",
Self::Paragraph => "Describe with a paragraph what is shown in the image.",
}
}
fn max_new_tokens(self) -> usize {
match self {
Self::Short => 32,
Self::Detailed => 56,
Self::Paragraph => 96,
}
}
}
impl Default for CaptionDetail {
fn default() -> Self {
Self::Paragraph
}
}
#[derive(Serialize)] #[derive(Serialize)]
pub struct CaptionModelStatus { pub struct CaptionModelStatus {
pub model_id: &'static str, pub model_id: &'static str,
@@ -45,6 +152,8 @@ pub struct CaptionModelProgress {
#[derive(Serialize)] #[derive(Serialize)]
pub struct CaptionRuntimeProbe { pub struct CaptionRuntimeProbe {
pub ready: bool, pub ready: bool,
pub acceleration: CaptionAcceleration,
pub detail: CaptionDetail,
pub tokenizer_vocab_size: usize, pub tokenizer_vocab_size: usize,
pub sessions: Vec<CaptionRuntimeSessionProbe>, pub sessions: Vec<CaptionRuntimeSessionProbe>,
} }
@@ -61,6 +170,7 @@ pub struct CaptionVisionProbe {
pub input_shape: Vec<i64>, pub input_shape: Vec<i64>,
pub output_shape: Vec<i64>, pub output_shape: Vec<i64>,
pub output_values: usize, pub output_values: usize,
pub acceleration: CaptionAcceleration,
} }
#[derive(Clone)] #[derive(Clone)]
@@ -71,9 +181,11 @@ struct TensorData {
pub struct FlorenceCaptioner { pub struct FlorenceCaptioner {
tokenizer: Tokenizer, tokenizer: Tokenizer,
caption_detail: CaptionDetail,
vision_session: Session, vision_session: Session,
embed_session: Session, embed_session: Session,
encoder_session: Session, encoder_session: Session,
decoder_prefill_session: Session,
decoder_session: Session, decoder_session: Session,
} }
@@ -81,6 +193,54 @@ pub fn model_dir(app_data_dir: &Path) -> PathBuf {
app_data_dir.join("models").join("florence-2-base-ft") app_data_dir.join("models").join("florence-2-base-ft")
} }
pub fn caption_acceleration(app_data_dir: &Path) -> CaptionAcceleration {
let path = app_data_dir.join(CAPTION_ACCELERATION_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return CaptionAcceleration::default();
};
match value.trim().to_ascii_lowercase().as_str() {
"cpu" => CaptionAcceleration::Cpu,
"directml" => CaptionAcceleration::Directml,
_ => CaptionAcceleration::Auto,
}
}
pub fn set_caption_acceleration(
app_data_dir: &Path,
acceleration: CaptionAcceleration,
) -> Result<CaptionAcceleration> {
let path = app_data_dir.join(CAPTION_ACCELERATION_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, acceleration.as_str())?;
CAPTION_SESSION_DIRTY.store(true, Ordering::Relaxed);
Ok(acceleration)
}
pub fn caption_detail(app_data_dir: &Path) -> CaptionDetail {
let path = app_data_dir.join(CAPTION_DETAIL_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return CaptionDetail::default();
};
match value.trim().to_ascii_lowercase().as_str() {
"short" => CaptionDetail::Short,
"paragraph" => CaptionDetail::Paragraph,
_ => CaptionDetail::Detailed,
}
}
pub fn set_caption_detail(app_data_dir: &Path, detail: CaptionDetail) -> Result<CaptionDetail> {
let path = app_data_dir.join(CAPTION_DETAIL_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, detail.as_str())?;
Ok(detail)
}
pub fn caption_model_status(app_data_dir: &Path) -> CaptionModelStatus { pub fn caption_model_status(app_data_dir: &Path) -> CaptionModelStatus {
let local_dir = model_dir(app_data_dir); let local_dir = model_dir(app_data_dir);
let missing_files = REQUIRED_FILES let missing_files = REQUIRED_FILES
@@ -133,9 +293,20 @@ pub fn prepare_caption_model_with_progress(
if let Some(parent) = destination.parent() { if let Some(parent) = destination.parent() {
std::fs::create_dir_all(parent)?; std::fs::create_dir_all(parent)?;
} }
let cached = repo.get(file)?; if ONNX_RUNTIME_FILES
std::fs::copy(cached, destination)?; .iter()
completed_files += 1; .any(|(runtime_file, _, _)| runtime_file == file)
{
download_onnx_runtime_files(&local_dir)?;
completed_files = REQUIRED_FILES
.iter()
.filter(|file| local_dir.join(file).exists())
.count();
} else {
let cached = repo.get(file)?;
std::fs::copy(cached, destination)?;
completed_files += 1;
}
emit_progress(CaptionModelProgress { emit_progress(CaptionModelProgress {
total_files: REQUIRED_FILES.len(), total_files: REQUIRED_FILES.len(),
completed_files, completed_files,
@@ -172,21 +343,40 @@ pub fn probe_caption_runtime(app_data_dir: &Path) -> Result<CaptionRuntimeProbe>
} }
let local_dir = model_dir(app_data_dir); let local_dir = model_dir(app_data_dir);
ensure_onnx_runtime(&local_dir)?;
let tokenizer = let tokenizer =
Tokenizer::from_file(local_dir.join("tokenizer.json")).map_err(anyhow::Error::msg)?; Tokenizer::from_file(local_dir.join("tokenizer.json")).map_err(anyhow::Error::msg)?;
let sessions = [ let acceleration = caption_acceleration(app_data_dir);
"onnx/vision_encoder_fp16.onnx",
// For the vision encoder (the only session that runs on the GPU EP) we
// create an actual ORT session so we can verify which EP was actually
// loaded, rather than just echoing the settings file.
let vision_session = probe_vision_session(
&local_dir.join("onnx/vision_encoder_fp16.onnx"),
acceleration,
)?;
// The remaining entries are DLLs and CPU-only text/decoder models — probe
// them with the lightweight file-size check as before.
let other_files = [
"onnxruntime/onnxruntime.dll",
"onnxruntime/onnxruntime_providers_shared.dll",
"onnxruntime/DirectML.dll",
"onnx/embed_tokens_fp16.onnx", "onnx/embed_tokens_fp16.onnx",
"onnx/encoder_model_q4.onnx", "onnx/encoder_model_q4.onnx",
"onnx/decoder_model_q4.onnx",
"onnx/decoder_model_merged_q4.onnx", "onnx/decoder_model_merged_q4.onnx",
] ];
.into_iter() let mut sessions = vec![vision_session];
.map(|file| probe_session(file, &local_dir.join(file))) for file in other_files {
.collect::<Result<Vec<_>>>()?; sessions.push(probe_session(file, &local_dir.join(file))?);
}
Ok(CaptionRuntimeProbe { Ok(CaptionRuntimeProbe {
ready: true, ready: true,
acceleration,
detail: caption_detail(app_data_dir),
tokenizer_vocab_size: tokenizer.get_vocab_size(false), tokenizer_vocab_size: tokenizer.get_vocab_size(false),
sessions, sessions,
}) })
@@ -202,11 +392,17 @@ pub fn probe_caption_vision(app_data_dir: &Path, image_path: &Path) -> Result<Ca
} }
let local_dir = model_dir(app_data_dir); let local_dir = model_dir(app_data_dir);
ensure_onnx_runtime(&local_dir)?;
let pixels = preprocess_image(image_path)?; let pixels = preprocess_image(image_path)?;
let input_shape = vec![1, 3, 768, 768]; let input_shape = vec![1, 3, 768, 768];
let input = Tensor::from_array(([1usize, 3, 768, 768], pixels.into_boxed_slice())) let input = Tensor::from_array(([1usize, 3, 768, 768], pixels.into_boxed_slice()))
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
let mut session = create_session(&local_dir.join("onnx/vision_encoder_fp16.onnx"))?; let acceleration = caption_acceleration(app_data_dir);
let mut session = create_session(
&local_dir.join("onnx/vision_encoder_fp16.onnx"),
acceleration,
true,
)?;
let outputs = session let outputs = session
.run(ort::inputs! { .run(ort::inputs! {
"pixel_values" => input "pixel_values" => input
@@ -220,6 +416,7 @@ pub fn probe_caption_vision(app_data_dir: &Path, image_path: &Path) -> Result<Ca
input_shape, input_shape,
output_shape: output_shape.to_vec(), output_shape: output_shape.to_vec(),
output_values: output_values.len(), output_values: output_values.len(),
acceleration,
}) })
} }
@@ -230,6 +427,7 @@ pub fn generate_caption(app_data_dir: &Path, image_path: &Path) -> Result<String
impl FlorenceCaptioner { impl FlorenceCaptioner {
pub fn new(app_data_dir: &Path) -> Result<Self> { pub fn new(app_data_dir: &Path) -> Result<Self> {
let started_at = Instant::now();
let status = caption_model_status(app_data_dir); let status = caption_model_status(app_data_dir);
if !status.ready { if !status.ready {
anyhow::bail!( anyhow::bail!(
@@ -239,48 +437,92 @@ impl FlorenceCaptioner {
} }
let local_dir = model_dir(app_data_dir); let local_dir = model_dir(app_data_dir);
ensure_onnx_runtime(&local_dir)?;
let tokenizer = let tokenizer =
Tokenizer::from_file(local_dir.join("tokenizer.json")).map_err(anyhow::Error::msg)?; Tokenizer::from_file(local_dir.join("tokenizer.json")).map_err(anyhow::Error::msg)?;
let caption_detail = caption_detail(app_data_dir);
let vision_session = create_session(&local_dir.join("onnx/vision_encoder_fp16.onnx"))?; let sessions_started_at = Instant::now();
let embed_session = create_session(&local_dir.join("onnx/embed_tokens_fp16.onnx"))?; let acceleration = caption_acceleration(app_data_dir);
let encoder_session = create_session(&local_dir.join("onnx/encoder_model_q4.onnx"))?; let vision_session = create_session(
let decoder_session = create_session(&local_dir.join("onnx/decoder_model_merged_q4.onnx"))?; &local_dir.join("onnx/vision_encoder_fp16.onnx"),
acceleration,
true,
)?;
let embed_session = create_session(
&local_dir.join("onnx/embed_tokens_fp16.onnx"),
acceleration,
false,
)?;
let encoder_session = create_session(
&local_dir.join("onnx/encoder_model_q4.onnx"),
acceleration,
false,
)?;
let decoder_prefill_session = create_session(
&local_dir.join("onnx/decoder_model_q4.onnx"),
acceleration,
false,
)?;
let decoder_session = create_session(
&local_dir.join("onnx/decoder_model_merged_q4.onnx"),
acceleration,
false,
)?;
println!(
"Florence sessions loaded in {:?} with {:?} acceleration (total init {:?})",
sessions_started_at.elapsed(),
acceleration,
started_at.elapsed()
);
Ok(Self { Ok(Self {
tokenizer, tokenizer,
caption_detail,
vision_session, vision_session,
embed_session, embed_session,
encoder_session, encoder_session,
decoder_prefill_session,
decoder_session, decoder_session,
}) })
} }
pub fn generate(&mut self, image_path: &Path) -> Result<String> { pub fn generate(&mut self, image_path: &Path) -> Result<String> {
let started_at = Instant::now();
println!("Florence caption started: {}", image_path.display());
let image_features = run_vision_encoder(&mut self.vision_session, image_path)?; let image_features = run_vision_encoder(&mut self.vision_session, image_path)?;
println!("Florence vision encoder done in {:?}", started_at.elapsed());
let prompt_ids = self let prompt_ids = self
.tokenizer .tokenizer
.encode("What does the image describe?", false) .encode(self.caption_detail.prompt(), false)
.map_err(anyhow::Error::msg)? .map_err(anyhow::Error::msg)?
.get_ids() .get_ids()
.iter() .iter()
.map(|id| i64::from(*id)) .map(|id| i64::from(*id))
.collect::<Vec<_>>(); .collect::<Vec<_>>();
let prompt_embeds = run_token_embedder(&mut self.embed_session, &prompt_ids)?; let prompt_embeds = run_token_embedder(&mut self.embed_session, &prompt_ids)?;
let encoder_embeds = concatenate_sequence_embeddings(&prompt_embeds, &image_features)?; println!(
"Florence token embeddings done in {:?}",
started_at.elapsed()
);
let encoder_embeds = concatenate_sequence_embeddings(&image_features, &prompt_embeds)?;
let encoder_attention_mask = vec![1_i64; encoder_embeds.shape[1] as usize]; let encoder_attention_mask = vec![1_i64; encoder_embeds.shape[1] as usize];
let encoder_hidden_states = run_encoder( let encoder_hidden_states = run_encoder(
&mut self.encoder_session, &mut self.encoder_session,
&encoder_embeds, &encoder_embeds,
&encoder_attention_mask, &encoder_attention_mask,
)?; )?;
println!("Florence encoder done in {:?}", started_at.elapsed());
let generated_ids = run_decoder( let generated_ids = run_decoder(
&mut self.decoder_prefill_session,
&mut self.decoder_session, &mut self.decoder_session,
&mut self.embed_session, &mut self.embed_session,
&encoder_hidden_states, &encoder_hidden_states,
&encoder_attention_mask, &encoder_attention_mask,
self.caption_detail.max_new_tokens(),
)?; )?;
println!("Florence decoder done in {:?}", started_at.elapsed());
let generated_u32 = generated_ids let generated_u32 = generated_ids
.into_iter() .into_iter()
@@ -330,6 +572,26 @@ fn probe_session(file: &'static str, path: &Path) -> Result<CaptionRuntimeSessio
], ],
vec!["logits".to_string(), "present_key_values".to_string()], vec!["logits".to_string(), "present_key_values".to_string()],
), ),
"onnxruntime/onnxruntime.dll" => (
vec!["OrtGetApiBase".to_string()],
vec!["ORT DirectML 1.24.2".to_string()],
),
"onnxruntime/onnxruntime_providers_shared.dll" => (
vec!["providers".to_string()],
vec!["shared provider bridge".to_string()],
),
"onnxruntime/DirectML.dll" => (
vec!["DirectX 12".to_string()],
vec!["DirectML 1.15.4".to_string()],
),
"onnx/decoder_model_q4.onnx" => (
vec![
"encoder_attention_mask".to_string(),
"encoder_hidden_states".to_string(),
"inputs_embeds".to_string(),
],
vec!["logits".to_string(), "present_key_values".to_string()],
),
_ => (Vec::new(), Vec::new()), _ => (Vec::new(), Vec::new()),
}; };
@@ -340,6 +602,111 @@ fn probe_session(file: &'static str, path: &Path) -> Result<CaptionRuntimeSessio
}) })
} }
/// Create an actual ORT session for the vision encoder and return a probe
/// entry that reflects which execution provider was successfully loaded.
/// This is the only session that can legitimately run on DirectML; all
/// text/decoder sessions intentionally use CPU only.
fn probe_vision_session(
path: &Path,
acceleration: CaptionAcceleration,
) -> Result<CaptionRuntimeSessionProbe> {
let metadata = std::fs::metadata(path)?;
if metadata.len() == 0 {
anyhow::bail!("{} is empty", path.display());
}
// Try to create the session with the requested acceleration. If DirectML
// was requested but fails we report what actually loaded.
let loaded_acceleration = match acceleration {
CaptionAcceleration::Cpu => {
create_session(path, CaptionAcceleration::Cpu, false)?;
CaptionAcceleration::Cpu
}
CaptionAcceleration::Auto => {
// `fail_silently` — session will fall back to CPU if DirectML
// is unavailable; we detect this by trying Directml explicitly.
let directml_ok = create_session(path, CaptionAcceleration::Directml, true).is_ok();
if directml_ok {
CaptionAcceleration::Directml
} else {
create_session(path, CaptionAcceleration::Cpu, false)?;
CaptionAcceleration::Cpu
}
}
CaptionAcceleration::Directml => {
create_session(path, CaptionAcceleration::Directml, true)?;
CaptionAcceleration::Directml
}
};
Ok(CaptionRuntimeSessionProbe {
file: "onnx/vision_encoder_fp16.onnx",
inputs: vec!["pixel_values".to_string()],
outputs: vec![format!("image_features [EP: {:?}]", loaded_acceleration)],
})
}
pub fn ensure_onnx_runtime(local_dir: &Path) -> Result<()> {
let dll_path = local_dir.join(ONNX_RUNTIME_DLL_FILE);
ORT_RUNTIME_INIT
.get_or_init(|| {
if !dll_path.exists() {
return Err(format!(
"ONNX Runtime DLL is missing: {}",
dll_path.display()
));
}
ort::environment::init_from(&dll_path)
.map_err(|error| error.to_string())?
.with_name("phokus-florence")
.commit();
Ok(())
})
.clone()
.map_err(anyhow::Error::msg)
}
fn download_onnx_runtime_files(local_dir: &Path) -> Result<()> {
if !cfg!(target_os = "windows") {
anyhow::bail!(
"Florence-2 ONNX Runtime download is currently configured for Windows builds"
);
}
for (destination_file, source_url, archive_path) in ONNX_RUNTIME_FILES {
let destination = local_dir.join(destination_file);
download_nuget_file(source_url, archive_path, &destination)?;
}
Ok(())
}
fn download_nuget_file(source_url: &str, archive_path: &str, destination: &Path) -> Result<()> {
let mut response = ureq::get(source_url)
.call()
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut bytes = Vec::new();
response
.body_mut()
.as_reader()
.read_to_end(&mut bytes)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut archive = zip::ZipArchive::new(Cursor::new(bytes))?;
let mut dll = archive.by_name(archive_path)?;
if let Some(parent) = destination.parent() {
std::fs::create_dir_all(parent)?;
}
let temp_destination = destination.with_extension("tmp");
{
let mut file = std::fs::File::create(&temp_destination)?;
std::io::copy(&mut dll, &mut file)?;
}
std::fs::rename(temp_destination, destination)?;
Ok(())
}
fn run_vision_encoder(session: &mut Session, image_path: &Path) -> Result<TensorData> { fn run_vision_encoder(session: &mut Session, image_path: &Path) -> Result<TensorData> {
let pixels = preprocess_image(image_path)?; let pixels = preprocess_image(image_path)?;
let input = Tensor::from_array(([1usize, 3, 768, 768], pixels.into_boxed_slice())) let input = Tensor::from_array(([1usize, 3, 768, 768], pixels.into_boxed_slice()))
@@ -387,24 +754,50 @@ fn run_encoder(
} }
fn run_decoder( fn run_decoder(
decoder_prefill_session: &mut Session,
decoder_session: &mut Session, decoder_session: &mut Session,
embed_session: &mut Session, embed_session: &mut Session,
encoder_hidden_states: &TensorData, encoder_hidden_states: &TensorData,
encoder_attention_mask: &[i64], encoder_attention_mask: &[i64],
max_new_tokens: usize,
) -> Result<Vec<i64>> { ) -> Result<Vec<i64>> {
const DECODER_LAYERS: usize = 6; const DECODER_LAYERS: usize = 6;
const DECODER_HEADS: usize = 12;
const HEAD_DIM: usize = 64;
const DECODER_START_TOKEN_ID: i64 = 2; const DECODER_START_TOKEN_ID: i64 = 2;
const FORCED_BOS_TOKEN_ID: i64 = 0;
const EOS_TOKEN_ID: i64 = 2; const EOS_TOKEN_ID: i64 = 2;
const MAX_NEW_TOKENS: usize = 32;
let mut generated = Vec::new(); let mut generated = Vec::new();
let mut next_input_id = DECODER_START_TOKEN_ID; let encoder_attention_mask_tensor = Tensor::from_array((
let mut past: Vec<TensorData> = Vec::new(); [1usize, encoder_attention_mask.len()],
let mut use_cache_branch = false; encoder_attention_mask.to_vec().into_boxed_slice(),
))
.map_err(|error| anyhow::anyhow!("{error}"))?;
let encoder_hidden_states_tensor = tensor_from_data(encoder_hidden_states)?;
let prefill_inputs_embeds = run_token_embedder(embed_session, &[DECODER_START_TOKEN_ID])?;
let prefill_inputs_embeds_tensor = tensor_from_data(&prefill_inputs_embeds)?;
let prefill_started_at = Instant::now();
let prefill_outputs = decoder_prefill_session
.run(ort::inputs! {
"encoder_attention_mask" => encoder_attention_mask_tensor,
"encoder_hidden_states" => encoder_hidden_states_tensor,
"inputs_embeds" => prefill_inputs_embeds_tensor
})
.map_err(|error| anyhow::anyhow!("{error}"))?;
println!(
"Florence decoder prefill done in {:?}",
prefill_started_at.elapsed()
);
let _prefill_logits = tensor_data(&prefill_outputs[0])?;
let mut next_input_id = FORCED_BOS_TOKEN_ID;
let encoder_kv = collect_present_key_values(&prefill_outputs, DECODER_LAYERS)?;
let mut decoder_kv = encoder_kv.clone();
for _ in 0..max_new_tokens {
if next_input_id == EOS_TOKEN_ID {
break;
}
generated.push(next_input_id);
for _ in 0..MAX_NEW_TOKENS {
let inputs_embeds = run_token_embedder(embed_session, &[next_input_id])?; let inputs_embeds = run_token_embedder(embed_session, &[next_input_id])?;
let encoder_attention_mask_tensor = Tensor::from_array(( let encoder_attention_mask_tensor = Tensor::from_array((
[1usize, encoder_attention_mask.len()], [1usize, encoder_attention_mask.len()],
@@ -413,9 +806,8 @@ fn run_decoder(
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
let encoder_hidden_states_tensor = tensor_from_data(encoder_hidden_states)?; let encoder_hidden_states_tensor = tensor_from_data(encoder_hidden_states)?;
let inputs_embeds_tensor = tensor_from_data(&inputs_embeds)?; let inputs_embeds_tensor = tensor_from_data(&inputs_embeds)?;
let use_cache_branch_tensor = let use_cache_branch_tensor = Tensor::from_array(([1usize], vec![true].into_boxed_slice()))
Tensor::from_array(([1usize], vec![use_cache_branch].into_boxed_slice())) .map_err(|error| anyhow::anyhow!("{error}"))?;
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut inputs = ort::inputs! { let mut inputs = ort::inputs! {
"encoder_attention_mask" => encoder_attention_mask_tensor, "encoder_attention_mask" => encoder_attention_mask_tensor,
@@ -424,92 +816,82 @@ fn run_decoder(
"use_cache_branch" => use_cache_branch_tensor "use_cache_branch" => use_cache_branch_tensor
}; };
if past.is_empty() { for layer in 0..DECODER_LAYERS {
for layer in 0..DECODER_LAYERS { push_tensor_input(
push_tensor_input( &mut inputs,
&mut inputs, format!("past_key_values.{layer}.decoder.key"),
format!("past_key_values.{layer}.decoder.key"), decoder_kv[layer * 4].clone(),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]), )?;
)?; push_tensor_input(
push_tensor_input( &mut inputs,
&mut inputs, format!("past_key_values.{layer}.decoder.value"),
format!("past_key_values.{layer}.decoder.value"), decoder_kv[layer * 4 + 1].clone(),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]), )?;
)?; push_tensor_input(
push_tensor_input( &mut inputs,
&mut inputs, format!("past_key_values.{layer}.encoder.key"),
format!("past_key_values.{layer}.encoder.key"), encoder_kv[layer * 4 + 2].clone(),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]), )?;
)?; push_tensor_input(
push_tensor_input( &mut inputs,
&mut inputs, format!("past_key_values.{layer}.encoder.value"),
format!("past_key_values.{layer}.encoder.value"), encoder_kv[layer * 4 + 3].clone(),
TensorData::zeros(vec![1, DECODER_HEADS as i64, 0, HEAD_DIM as i64]), )?;
)?;
}
} else {
for layer in 0..DECODER_LAYERS {
for cache_name in [
"decoder.key",
"decoder.value",
"encoder.key",
"encoder.value",
] {
let past_index = layer * 4
+ match cache_name {
"decoder.key" => 0,
"decoder.value" => 1,
"encoder.key" => 2,
"encoder.value" => 3,
_ => unreachable!(),
};
push_tensor_input(
&mut inputs,
format!("past_key_values.{layer}.{cache_name}"),
past[past_index].clone(),
)?;
}
}
} }
let outputs = decoder_session let outputs = decoder_session
.run(inputs) .run(inputs)
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
let logits = tensor_data(&outputs["logits"])?; let logits = tensor_data(&outputs["logits"])?;
let token_id = argmax_last_token(&logits)?; next_input_id = argmax_last_token(&logits)?;
if token_id == EOS_TOKEN_ID { decoder_kv = collect_present_key_values(&outputs, DECODER_LAYERS)?;
break;
}
generated.push(token_id);
next_input_id = token_id;
past.clear();
for layer in 0..DECODER_LAYERS {
for cache_name in [
"decoder.key",
"decoder.value",
"encoder.key",
"encoder.value",
] {
past.push(tensor_data(
&outputs[format!("present.{layer}.{cache_name}").as_str()],
)?);
}
}
use_cache_branch = true;
} }
println!("Florence decoder produced {} token(s)", generated.len());
Ok(generated) Ok(generated)
} }
fn create_session(path: &Path) -> Result<Session> { fn create_session(
path: &Path,
acceleration: CaptionAcceleration,
allow_directml: bool,
) -> Result<Session> {
let builder = Session::builder().map_err(|error| anyhow::anyhow!("{error}"))?; let builder = Session::builder().map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder let builder = builder
.with_optimization_level(GraphOptimizationLevel::Level3) .with_optimization_level(GraphOptimizationLevel::Level3)
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
let mut builder = builder let use_directml = allow_directml
&& matches!(
acceleration,
CaptionAcceleration::Auto | CaptionAcceleration::Directml
);
let builder = builder
.with_memory_pattern(!use_directml)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_parallel_execution(false)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_intra_threads(1) .with_intra_threads(1)
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
let mut builder = match acceleration {
CaptionAcceleration::Cpu => builder,
CaptionAcceleration::Auto if use_directml => builder
.with_execution_providers([ep::DirectML::default().build().fail_silently()])
.unwrap_or_else(|error| error.recover()),
CaptionAcceleration::Directml if use_directml => builder
.with_execution_providers([ep::DirectML::default().build().error_on_failure()])
.map_err(|error| anyhow::anyhow!("{error}"))?,
CaptionAcceleration::Auto | CaptionAcceleration::Directml => {
// `allow_directml` is false for the 4 text/decoder sessions —
// they intentionally run on CPU regardless of the setting.
println!(
"Florence: using CPU for {} (DirectML disabled for this session type)",
path.display()
);
builder
}
};
let session = builder let session = builder
.commit_from_file(path) .commit_from_file(path)
.map_err(|error| anyhow::anyhow!("{error}"))?; .map_err(|error| anyhow::anyhow!("{error}"))?;
@@ -537,6 +919,24 @@ fn concatenate_sequence_embeddings(text: &TensorData, image: &TensorData) -> Res
}) })
} }
fn collect_present_key_values(
outputs: &SessionOutputs<'_>,
layers: usize,
) -> Result<Vec<TensorData>> {
let expected = layers * 4;
if outputs.len() < expected + 1 {
anyhow::bail!(
"Decoder returned {} output(s), expected at least {}",
outputs.len(),
expected + 1
);
}
(1..=expected)
.map(|index| tensor_data(&outputs[index]))
.collect::<Result<Vec<_>>>()
}
fn tensor_data(value: &ort::value::DynValue) -> Result<TensorData> { fn tensor_data(value: &ort::value::DynValue) -> Result<TensorData> {
let (shape, values) = value let (shape, values) = value
.try_extract_tensor::<f32>() .try_extract_tensor::<f32>()
@@ -608,13 +1008,3 @@ fn preprocess_image(image_path: &Path) -> Result<Vec<f32>> {
Ok(pixel_values) Ok(pixel_values)
} }
impl TensorData {
fn zeros(shape: Vec<i64>) -> Self {
let values_len = shape.iter().map(|value| (*value).max(0) as usize).product();
Self {
shape,
values: vec![0.0; values_len],
}
}
}
+759 -20
View File
@@ -1,7 +1,12 @@
use crate::captioner::{self, CaptionModelStatus, CaptionRuntimeProbe, CaptionVisionProbe}; use crate::captioner::{
use crate::db::{self, DbPool, Folder, FolderJobProgress, ImageRecord}; self, CaptionAcceleration, CaptionDetail, CaptionModelStatus, CaptionRuntimeProbe,
CaptionVisionProbe,
};
use crate::db::{self, DbPool, ExploreTagEntry, Folder, FolderJobProgress, ImageRecord, ImageTag};
use crate::embedder; use crate::embedder;
use crate::hnsw_index;
use crate::indexer; use crate::indexer;
use crate::tagger::{self, TaggerAcceleration, TaggerModelStatus, TaggerRuntimeProbe};
use crate::vector; use crate::vector;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::path::PathBuf; use std::path::PathBuf;
@@ -17,12 +22,21 @@ pub struct ImagesPage {
pub limit: i64, pub limit: i64,
} }
#[derive(Serialize)]
pub struct SimilarImagesPage {
pub images: Vec<ImageRecord>,
pub offset: usize,
pub limit: usize,
pub has_more: bool,
}
#[derive(Deserialize)] #[derive(Deserialize)]
pub struct GetImagesParams { pub struct GetImagesParams {
pub folder_id: Option<i64>, pub folder_id: Option<i64>,
pub search: Option<String>, pub search: Option<String>,
pub media_kind: Option<String>, pub media_kind: Option<String>,
pub favorites_only: Option<bool>, pub favorites_only: Option<bool>,
pub rating_min: Option<i64>,
pub embedding_failed_only: Option<bool>, pub embedding_failed_only: Option<bool>,
pub sort: Option<String>, pub sort: Option<String>,
pub offset: Option<i64>, pub offset: Option<i64>,
@@ -39,12 +53,29 @@ pub struct UpdateImageDetailsParams {
#[derive(Deserialize)] #[derive(Deserialize)]
pub struct FindSimilarImagesParams { pub struct FindSimilarImagesParams {
pub image_id: i64, pub image_id: i64,
pub folder_id: Option<i64>,
pub offset: Option<usize>,
pub limit: Option<usize>,
pub threshold: Option<f32>,
}
#[derive(Deserialize)]
pub struct FindSimilarByRegionParams {
pub image_id: i64,
/// Normalized crop rect (0.01.0).
pub crop_x: f32,
pub crop_y: f32,
pub crop_w: f32,
pub crop_h: f32,
pub folder_id: Option<i64>,
pub offset: Option<usize>,
pub limit: Option<usize>, pub limit: Option<usize>,
} }
#[derive(Deserialize)] #[derive(Deserialize)]
pub struct DebugSimilarImagesParams { pub struct DebugSimilarImagesParams {
pub image_id: i64, pub image_id: i64,
pub folder_id: Option<i64>,
pub limit: Option<usize>, pub limit: Option<usize>,
} }
@@ -72,6 +103,26 @@ pub struct QueueCaptionJobsParams {
pub image_id: Option<i64>, pub image_id: Option<i64>,
} }
#[derive(Deserialize)]
pub struct ClearCaptionJobsParams {
pub folder_id: Option<i64>,
}
#[derive(Deserialize)]
pub struct ResetCaptionsParams {
pub folder_id: Option<i64>,
}
#[derive(Deserialize)]
pub struct SetCaptionAccelerationParams {
pub acceleration: CaptionAcceleration,
}
#[derive(Deserialize)]
pub struct SetCaptionDetailParams {
pub detail: CaptionDetail,
}
#[derive(Deserialize)] #[derive(Deserialize)]
pub struct ProbeCaptionImageParams { pub struct ProbeCaptionImageParams {
pub image_id: i64, pub image_id: i64,
@@ -88,9 +139,56 @@ pub struct SemanticSearchParams {
pub folder_id: Option<i64>, pub folder_id: Option<i64>,
pub media_kind: Option<String>, pub media_kind: Option<String>,
pub favorites_only: Option<bool>, pub favorites_only: Option<bool>,
pub rating_min: Option<i64>,
pub limit: Option<usize>, pub limit: Option<usize>,
} }
#[derive(Deserialize)]
pub struct TagSearchParams {
pub query: String,
pub folder_id: Option<i64>,
pub media_kind: Option<String>,
pub favorites_only: Option<bool>,
pub rating_min: Option<i64>,
pub limit: Option<usize>,
}
#[derive(Deserialize)]
pub struct GetExploreTagsParams {
pub folder_id: Option<i64>,
pub limit: Option<usize>,
}
#[derive(Deserialize)]
pub struct SearchTagsAutocompleteParams {
pub query: String,
pub folder_id: Option<i64>,
pub limit: Option<usize>,
}
#[derive(Serialize, Deserialize)]
pub struct DuplicateGroup {
pub file_hash: String,
pub file_size: u64,
pub images: Vec<ImageRecord>,
}
#[derive(Serialize)]
pub struct DuplicateScanCache {
pub groups: Vec<DuplicateGroup>,
pub scanned_at: i64,
}
#[derive(Deserialize)]
pub struct DeleteImagesFromDiskParams {
pub image_ids: Vec<i64>,
}
#[derive(Deserialize)]
pub struct GetImagesByIdsParams {
pub image_ids: Vec<i64>,
}
#[tauri::command] #[tauri::command]
pub async fn add_folder( pub async fn add_folder(
app: AppHandle, app: AppHandle,
@@ -161,6 +259,7 @@ pub async fn get_images(
let search = params.search.as_deref(); let search = params.search.as_deref();
let media_kind = params.media_kind.as_deref(); let media_kind = params.media_kind.as_deref();
let favorites_only = params.favorites_only.unwrap_or(false); let favorites_only = params.favorites_only.unwrap_or(false);
let rating_min = params.rating_min.unwrap_or(0);
let embedding_failed_only = params.embedding_failed_only.unwrap_or(false); let embedding_failed_only = params.embedding_failed_only.unwrap_or(false);
let total = db::count_images( let total = db::count_images(
@@ -169,6 +268,7 @@ pub async fn get_images(
search, search,
media_kind, media_kind,
favorites_only, favorites_only,
rating_min,
embedding_failed_only, embedding_failed_only,
) )
.map_err(|e| e.to_string())?; .map_err(|e| e.to_string())?;
@@ -179,6 +279,7 @@ pub async fn get_images(
search, search,
media_kind, media_kind,
favorites_only, favorites_only,
rating_min,
embedding_failed_only, embedding_failed_only,
sort, sort,
offset, offset,
@@ -228,16 +329,102 @@ pub async fn reindex_folder(
pub async fn find_similar_images( pub async fn find_similar_images(
db: State<'_, DbState>, db: State<'_, DbState>,
params: FindSimilarImagesParams, params: FindSimilarImagesParams,
) -> Result<Vec<ImageRecord>, String> { ) -> Result<SimilarImagesPage, String> {
let conn = db.get().map_err(|e| e.to_string())?; let conn = db.get().map_err(|e| e.to_string())?;
let limit = params.limit.unwrap_or(32); let limit = params.limit.unwrap_or(32);
let offset = params.offset.unwrap_or(0);
let threshold = params.threshold.unwrap_or(0.24);
if !vector::has_image_vector(&conn, params.image_id).map_err(|e| e.to_string())? { if !vector::has_image_vector(&conn, params.image_id).map_err(|e| e.to_string())? {
db::repair_embedding_consistency(&conn).map_err(|e| e.to_string())?; db::repair_embedding_consistency(&conn).map_err(|e| e.to_string())?;
return Ok(Vec::new()); return Ok(SimilarImagesPage {
images: Vec::new(),
offset,
limit,
has_more: false,
});
} }
let image_ids = let matches = hnsw_index::find_similar_image_matches(
vector::find_similar_image_ids(&conn, params.image_id, limit).map_err(|e| e.to_string())?; &conn,
db::get_images_by_ids(&conn, &image_ids).map_err(|e| e.to_string()) params.image_id,
params.folder_id,
threshold,
offset,
limit + 1,
)
.map_err(|e| e.to_string())?;
let has_more = matches.len() > limit;
let image_ids = matches
.into_iter()
.take(limit)
.map(|(image_id, _)| image_id)
.collect::<Vec<_>>();
let images = db::get_images_by_ids(&conn, &image_ids).map_err(|e| e.to_string())?;
Ok(SimilarImagesPage {
images,
offset,
limit,
has_more,
})
}
#[tauri::command]
pub async fn find_similar_by_region(
db: State<'_, DbState>,
params: FindSimilarByRegionParams,
) -> Result<SimilarImagesPage, String> {
let conn = db.get().map_err(|e| e.to_string())?;
let limit = params.limit.unwrap_or(32);
let offset = params.offset.unwrap_or(0);
// Look up the source image path
let image = db::get_image_by_id(&conn, params.image_id).map_err(|e| e.to_string())?;
let image_path = std::path::Path::new(&image.path);
// Embed the cropped region in-memory (no temp file needed)
let embedder = embedder::ClipImageEmbedder::new().map_err(|e| e.to_string())?;
let embedding = embedder
.embed_image_crop(
image_path,
params.crop_x,
params.crop_y,
params.crop_w,
params.crop_h,
)
.map_err(|e| e.to_string())?;
// Search for similar images using the crop embedding
let image_ids = match params.folder_id {
Some(folder_id) => vector::search_image_ids_by_embedding_in_folder(
&conn,
&embedding,
folder_id,
Some(params.image_id),
offset + limit + 1,
)
.map_err(|e| e.to_string())?,
None => {
let mut ids = vector::search_image_ids_by_embedding(&conn, &embedding, offset + limit + 1)
.map_err(|e| e.to_string())?;
// Exclude the source image from global results
ids.retain(|&id| id != params.image_id);
ids
}
};
let has_more = image_ids.len() > offset + limit;
let page_ids = image_ids
.into_iter()
.skip(offset)
.take(limit)
.collect::<Vec<_>>();
let images = db::get_images_by_ids(&conn, &page_ids).map_err(|e| e.to_string())?;
Ok(SimilarImagesPage {
images,
offset,
limit,
has_more,
})
} }
#[derive(Serialize)] #[derive(Serialize)]
@@ -258,7 +445,8 @@ pub async fn debug_similar_images(
let vector_count = vector::count_image_vectors(&conn).map_err(|e| e.to_string())?; let vector_count = vector::count_image_vectors(&conn).map_err(|e| e.to_string())?;
let has_vector = vector::has_image_vector(&conn, params.image_id).map_err(|e| e.to_string())?; let has_vector = vector::has_image_vector(&conn, params.image_id).map_err(|e| e.to_string())?;
let similar_ids = if has_vector { let similar_ids = if has_vector {
vector::find_similar_image_ids(&conn, params.image_id, limit).map_err(|e| e.to_string())? vector::find_similar_image_ids(&conn, params.image_id, limit, params.folder_id)
.map_err(|e| e.to_string())?
} else { } else {
Vec::new() Vec::new()
}; };
@@ -301,10 +489,31 @@ pub async fn semantic_search_images(
if params.favorites_only.unwrap_or(false) { if params.favorites_only.unwrap_or(false) {
images.retain(|image| image.favorite); images.retain(|image| image.favorite);
} }
if let Some(rating_min) = params.rating_min {
images.retain(|image| image.rating >= rating_min);
}
Ok(images) Ok(images)
} }
#[tauri::command]
pub async fn search_images_by_tag(
db: State<'_, DbState>,
params: TagSearchParams,
) -> Result<Vec<ImageRecord>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::search_images_by_tag(
&conn,
&params.query,
params.folder_id,
params.media_kind.as_deref(),
params.favorites_only.unwrap_or(false),
params.rating_min.unwrap_or(0),
params.limit.unwrap_or(64),
)
.map_err(|e| e.to_string())
}
#[tauri::command] #[tauri::command]
pub async fn set_generated_caption( pub async fn set_generated_caption(
db: State<'_, DbState>, db: State<'_, DbState>,
@@ -332,6 +541,36 @@ pub async fn get_caption_model_status(app: AppHandle) -> Result<CaptionModelStat
Ok(captioner::caption_model_status(&app_dir)) Ok(captioner::caption_model_status(&app_dir))
} }
#[tauri::command]
pub async fn get_caption_acceleration(app: AppHandle) -> Result<CaptionAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(captioner::caption_acceleration(&app_dir))
}
#[tauri::command]
pub async fn set_caption_acceleration(
app: AppHandle,
params: SetCaptionAccelerationParams,
) -> Result<CaptionAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
captioner::set_caption_acceleration(&app_dir, params.acceleration).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_caption_detail(app: AppHandle) -> Result<CaptionDetail, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(captioner::caption_detail(&app_dir))
}
#[tauri::command]
pub async fn set_caption_detail(
app: AppHandle,
params: SetCaptionDetailParams,
) -> Result<CaptionDetail, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
captioner::set_caption_detail(&app_dir, params.detail).map_err(|e| e.to_string())
}
#[tauri::command] #[tauri::command]
pub async fn prepare_caption_model(app: AppHandle) -> Result<CaptionModelStatus, String> { pub async fn prepare_caption_model(app: AppHandle) -> Result<CaptionModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?; let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
@@ -441,11 +680,31 @@ pub async fn queue_caption_jobs(
} }
} }
#[tauri::command]
pub async fn clear_caption_jobs(
db: State<'_, DbState>,
params: ClearCaptionJobsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::clear_caption_jobs(&conn, params.folder_id).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn reset_generated_captions(
db: State<'_, DbState>,
params: ResetCaptionsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::reset_generated_captions(&conn, params.folder_id).map_err(|e| e.to_string())
}
#[derive(Serialize, Deserialize)] #[derive(Serialize, Deserialize)]
pub struct TagCloudEntry { pub struct TagCloudEntry {
pub count: usize, pub count: usize,
pub representative_image_id: i64, pub representative_image_id: i64,
pub thumbnail_path: Option<String>, pub thumbnail_path: Option<String>,
#[serde(default)]
pub image_ids: Vec<i64>,
} }
fn fnv_hash_ids(ids: &[i64]) -> u64 { fn fnv_hash_ids(ids: &[i64]) -> u64 {
@@ -494,7 +753,11 @@ pub async fn get_tag_cloud(
.map_err(|e| e.to_string())? .map_err(|e| e.to_string())?
{ {
if let Ok(entries) = serde_json::from_str::<Vec<TagCloudEntry>>(&json) { if let Ok(entries) = serde_json::from_str::<Vec<TagCloudEntry>>(&json) {
return Ok(entries); // Reject cache entries written before image_ids were tracked — they all
// have empty image_ids which causes "No media found" when a cluster is opened.
if entries.iter().all(|e| !e.image_ids.is_empty()) {
return Ok(entries);
}
} }
} }
} }
@@ -522,6 +785,13 @@ pub async fn get_tag_cloud(
} }
let centroid = &centroids[ci]; let centroid = &centroids[ci];
let cluster_ids = points
.iter()
.enumerate()
.filter(|(i, _)| assignments[*i] == ci)
.map(|(i, _)| ids[i])
.collect::<Vec<_>>();
let best_id = points let best_id = points
.iter() .iter()
.enumerate() .enumerate()
@@ -539,6 +809,7 @@ pub async fn get_tag_cloud(
count, count,
representative_image_id: best_id, representative_image_id: best_id,
thumbnail_path, thumbnail_path,
image_ids: cluster_ids,
}); });
} }
@@ -550,6 +821,224 @@ pub async fn get_tag_cloud(
Ok(entries) Ok(entries)
} }
#[tauri::command]
pub async fn get_explore_tags(
db: State<'_, DbState>,
params: GetExploreTagsParams,
) -> Result<Vec<ExploreTagEntry>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_explore_tags(&conn, params.folder_id, params.limit.unwrap_or(48)).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn search_tags_autocomplete(
db: State<'_, DbState>,
params: SearchTagsAutocompleteParams,
) -> Result<Vec<ExploreTagEntry>, String> {
if params.query.trim().is_empty() {
return Ok(vec![]);
}
let conn = db.get().map_err(|e| e.to_string())?;
db::search_tags_autocomplete(&conn, &params.query, params.folder_id, params.limit.unwrap_or(10))
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn find_duplicates(
app: AppHandle,
db: State<'_, DbState>,
folder_id: Option<i64>,
) -> Result<Vec<DuplicateGroup>, String> {
let records = {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_all_image_paths(&conn, folder_id).map_err(|e| e.to_string())?
};
let total = records.len();
let _ = app.emit("duplicate_scan_progress", (0usize, total));
// Two-phase detection. No full-file read at any point.
//
// Phase 1 — stat:
// Read only file metadata (size). Files with a unique size cannot be
// duplicates; discard them immediately. Zero file content read.
//
// Phase 2 — sample hash:
// For each size-matched candidate, read four evenly-spaced 16 KB windows
// (head, 33%, 66%, tail) and hash them together. Total I/O per file is
// capped at 64 KB regardless of how large the file is.
//
// For small files (≤ 64 KB) the windows cover the whole file, so the
// hash is exact. For large files, matching all four windows at different
// offsets is effectively impossible for natural photo/video content unless
// the files are genuinely identical — eliminating the need for a full read.
let app_hash = app.clone();
let pairs: Vec<(u64, u64, i64)> = tokio::task::spawn_blocking(move || {
use memmap2::Mmap;
use rayon::prelude::*;
use std::collections::HashMap;
use std::sync::atomic::{AtomicUsize, Ordering};
use xxhash_rust::xxh3::{xxh3_64, Xxh3};
const WINDOW: usize = 16 * 1024; // 16 KB per sample window
const N: usize = 4; // windows at 0%, 33%, 66%, 100%
const COVERED: usize = WINDOW * N; // 64 KB total; also full-coverage threshold
// Hash four evenly-spaced 16 KB windows. For files ≤ 64 KB this is
// equivalent to hashing the entire file.
fn sample(mmap: &[u8]) -> u64 {
if mmap.len() <= COVERED {
return xxh3_64(mmap);
}
let mut h = Xxh3::new();
for i in 0..N {
let pos = (mmap.len() - WINDOW) * i / (N - 1);
h.update(&mmap[pos..pos + WINDOW]);
}
h.digest()
}
// ── Phase 1: stat ─────────────────────────────────────────────────
let sized: Vec<(i64, String, u64)> = records
.par_iter()
.filter_map(|r| {
let size = std::fs::metadata(&r.path).ok()?.len();
if size == 0 { return None; }
Some((r.id, r.path.clone(), size))
})
.collect();
let mut by_size: HashMap<u64, Vec<usize>> = HashMap::new();
for (i, (_, _, size)) in sized.iter().enumerate() {
by_size.entry(*size).or_default().push(i);
}
let candidates: Vec<usize> = by_size
.into_values()
.filter(|g| g.len() > 1)
.flatten()
.collect();
if candidates.is_empty() {
return vec![];
}
// ── Phase 2: sample hash ──────────────────────────────────────────
let c_total = candidates.len();
let _ = app_hash.emit("duplicate_scan_progress", (0usize, c_total));
let counter = AtomicUsize::new(0);
candidates
.par_iter()
.filter_map(|&idx| {
let (id, path, size) = &sized[idx];
let file = std::fs::File::open(path).ok()?;
// SAFETY: read-only; no external truncation expected during scan.
let mmap = unsafe { Mmap::map(&file).ok()? };
let hash = sample(&mmap);
let done = counter.fetch_add(1, Ordering::Relaxed) + 1;
if done % 100 == 0 || done == c_total {
let _ = app_hash.emit("duplicate_scan_progress", (done, c_total));
}
Some((hash, *size, *id))
})
.collect()
})
.await
.map_err(|e| e.to_string())?;
let mut size_hash_map: std::collections::HashMap<(u64, u64), Vec<i64>> = std::collections::HashMap::new();
for (hash, file_size, id) in pairs {
size_hash_map.entry((hash, file_size)).or_default().push(id);
}
// Resolve image records for each duplicate group
let conn = db.get().map_err(|e| e.to_string())?;
let mut groups: Vec<DuplicateGroup> = size_hash_map
.into_iter()
.filter(|(_, ids)| ids.len() > 1)
.filter_map(|((hash, file_size), ids)| {
let images = db::get_images_by_ids(&conn, &ids).ok()?;
Some(DuplicateGroup {
file_hash: format!("{:016x}", hash),
file_size,
images,
})
})
.collect();
// Largest duplicates first — wastes the most space
groups.sort_by(|a, b| b.file_size.cmp(&a.file_size));
// Persist results so they survive restart — best-effort, ignore errors.
let folder_scope = match folder_id {
Some(id) => format!("folder:{}", id),
None => "all".to_string(),
};
if let Ok(json) = serde_json::to_string(&groups) {
let _ = db::set_duplicate_scan_cache(&conn, &folder_scope, &json);
}
Ok(groups)
}
#[tauri::command]
pub async fn load_duplicate_scan_cache(
db: State<'_, DbState>,
folder_id: Option<i64>,
) -> Result<Option<DuplicateScanCache>, String> {
let folder_scope = match folder_id {
Some(id) => format!("folder:{}", id),
None => "all".to_string(),
};
let conn = db.get().map_err(|e| e.to_string())?;
match db::get_duplicate_scan_cache(&conn, &folder_scope).map_err(|e| e.to_string())? {
Some((json, scanned_at)) => {
let groups: Vec<DuplicateGroup> =
serde_json::from_str(&json).map_err(|e| e.to_string())?;
Ok(Some(DuplicateScanCache { groups, scanned_at }))
}
None => Ok(None),
}
}
#[tauri::command]
pub async fn delete_images_from_disk(
db: State<'_, DbState>,
params: DeleteImagesFromDiskParams,
) -> Result<usize, String> {
if params.image_ids.is_empty() {
return Ok(0);
}
let conn = db.get().map_err(|e| e.to_string())?;
// Collect paths before deleting DB rows
let records = db::get_all_image_paths(&conn, None).map_err(|e| e.to_string())?;
let id_set: std::collections::HashSet<i64> = params.image_ids.iter().copied().collect();
let paths: Vec<String> = records
.into_iter()
.filter(|r| id_set.contains(&r.id))
.map(|r| r.path)
.collect();
db::delete_images_by_ids(&conn, &params.image_ids).map_err(|e| e.to_string())?;
let mut deleted = 0usize;
for path in &paths {
if std::fs::remove_file(path).is_ok() {
deleted += 1;
}
}
Ok(deleted)
}
#[tauri::command]
pub async fn get_images_by_ids(
db: State<'_, DbState>,
params: GetImagesByIdsParams,
) -> Result<Vec<ImageRecord>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_images_by_ids(&conn, &params.image_ids).map_err(|e| e.to_string())
}
// ── k-means with cosine similarity (all vectors assumed to be unit-normalized) ── // ── k-means with cosine similarity (all vectors assumed to be unit-normalized) ──
fn dot(a: &[f32], b: &[f32]) -> f32 { fn dot(a: &[f32], b: &[f32]) -> f32 {
@@ -670,11 +1159,27 @@ pub struct FolderWorkerStates {
pub metadata_paused: bool, pub metadata_paused: bool,
pub embedding_paused: bool, pub embedding_paused: bool,
pub caption_paused: bool, pub caption_paused: bool,
pub tagging_paused: bool,
} }
#[tauri::command] #[tauri::command]
pub async fn set_worker_paused(worker: String, folder_id: i64, paused: bool) -> Result<(), String> { pub async fn set_worker_paused(
db: State<'_, DbState>,
worker: String,
folder_id: i64,
paused: bool,
) -> Result<(), String> {
indexer::set_worker_paused(&worker, folder_id, paused); indexer::set_worker_paused(&worker, folder_id, paused);
if worker == "caption" && paused {
let conn = db.get().map_err(|e| e.to_string())?;
db::requeue_processing_caption_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
if worker == "tagging" && paused {
let conn = db.get().map_err(|e| e.to_string())?;
db::requeue_processing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
Ok(()) Ok(())
} }
@@ -684,23 +1189,257 @@ pub async fn get_worker_states(folder_ids: Vec<i64>) -> Result<Vec<FolderWorkerS
Ok(folder_ids Ok(folder_ids
.into_iter() .into_iter()
.map(|folder_id| { .map(|folder_id| {
let state = let state = states
states .get(&folder_id)
.get(&folder_id) .copied()
.copied() .unwrap_or(indexer::FolderWorkerPausedState {
.unwrap_or(indexer::FolderWorkerPausedState { thumbnail: false,
thumbnail: false, metadata: false,
metadata: false, embedding: false,
embedding: false, caption: false,
caption: false, tagging: false,
}); });
FolderWorkerStates { FolderWorkerStates {
folder_id, folder_id,
thumbnail_paused: state.thumbnail, thumbnail_paused: state.thumbnail,
metadata_paused: state.metadata, metadata_paused: state.metadata,
embedding_paused: state.embedding, embedding_paused: state.embedding,
caption_paused: state.caption, caption_paused: state.caption,
tagging_paused: state.tagging,
} }
}) })
.collect()) .collect())
} }
// ---------------------------------------------------------------------------
// Tagger commands
// ---------------------------------------------------------------------------
#[derive(Deserialize)]
pub struct SetTaggerAccelerationParams {
pub acceleration: TaggerAcceleration,
}
#[derive(Deserialize)]
pub struct SetTaggerThresholdParams {
pub threshold: f32,
}
#[derive(Deserialize)]
pub struct SetTaggerBatchSizeParams {
pub batch_size: usize,
}
#[derive(Deserialize)]
pub struct QueueTaggingJobsParams {
pub folder_id: Option<i64>,
pub folder_ids: Option<Vec<i64>>,
pub image_id: Option<i64>,
}
#[derive(Deserialize)]
pub struct ClearTaggingJobsParams {
pub folder_id: Option<i64>,
pub folder_ids: Option<Vec<i64>>,
}
#[derive(Deserialize)]
pub struct GetImageTagsParams {
pub image_id: i64,
}
#[derive(Deserialize)]
pub struct AddUserTagParams {
pub image_id: i64,
pub tag: String,
}
#[derive(Deserialize)]
pub struct RemoveTagParams {
pub tag_id: i64,
}
#[tauri::command]
pub async fn get_tagger_model_status(app: AppHandle) -> Result<TaggerModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_model_status(&app_dir))
}
#[tauri::command]
pub async fn get_tagger_acceleration(app: AppHandle) -> Result<TaggerAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_acceleration(&app_dir))
}
#[tauri::command]
pub async fn set_tagger_acceleration(
app: AppHandle,
params: SetTaggerAccelerationParams,
) -> Result<TaggerAcceleration, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tagger::set_tagger_acceleration(&app_dir, params.acceleration).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn probe_tagger_runtime(app: AppHandle) -> Result<TaggerRuntimeProbe, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tauri::async_runtime::spawn_blocking(move || tagger::probe_tagger_runtime(&app_dir))
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_tagger_threshold(app: AppHandle) -> Result<f32, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_threshold(&app_dir))
}
#[tauri::command]
pub async fn set_tagger_threshold(
app: AppHandle,
params: SetTaggerThresholdParams,
) -> Result<f32, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tagger::set_tagger_threshold(&app_dir, params.threshold).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_tagger_batch_size(app: AppHandle) -> Result<usize, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
Ok(tagger::tagger_batch_size(&app_dir))
}
#[tauri::command]
pub async fn set_tagger_batch_size(
app: AppHandle,
params: SetTaggerBatchSizeParams,
) -> Result<usize, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tagger::set_tagger_batch_size(&app_dir, params.batch_size).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn prepare_tagger_model(app: AppHandle) -> Result<TaggerModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tauri::async_runtime::spawn_blocking(move || {
let app = app.clone();
tagger::prepare_tagger_model_with_progress(&app_dir, move |progress| {
let _ = app.emit("tagger-model-progress", progress);
})
})
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn delete_tagger_model(app: AppHandle) -> Result<TaggerModelStatus, String> {
let app_dir = app.path().app_data_dir().map_err(|e| e.to_string())?;
tauri::async_runtime::spawn_blocking(move || tagger::delete_tagger_model(&app_dir))
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn queue_tagging_jobs(
app: AppHandle,
db: State<'_, DbState>,
params: QueueTaggingJobsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
let requested_folder_ids = params.folder_ids.unwrap_or_default();
let (total, folder_ids) = match (params.folder_id, params.image_id, requested_folder_ids.is_empty()) {
(_, Some(image_id), _) => {
db::enqueue_tagging_job(&conn, image_id).map_err(|e| e.to_string())?;
// Look up just this image's folder_id rather than fetching all folders
let image = db::get_image_by_id(&conn, image_id).map_err(|e| e.to_string())?;
(1usize, vec![image.folder_id])
}
(Some(folder_id), None, _) => {
let n = db::enqueue_missing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
(n, vec![folder_id])
}
(None, None, false) => {
let mut total = 0usize;
for &folder_id in &requested_folder_ids {
total += db::enqueue_missing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
(total, requested_folder_ids)
}
(None, None, true) => {
let folders = db::get_folders(&conn).map_err(|e| e.to_string())?;
let folder_ids: Vec<i64> = folders.iter().map(|f| f.id).collect();
let mut total = 0usize;
for &folder_id in &folder_ids {
total += db::enqueue_missing_tagging_jobs_for_folder(&conn, folder_id)
.map_err(|e| e.to_string())?;
}
(total, folder_ids)
}
};
drop(conn);
indexer::emit_folder_job_progress(&app, db.inner(), &folder_ids, true);
Ok(total)
}
#[tauri::command]
pub async fn clear_tagging_jobs(
app: AppHandle,
db: State<'_, DbState>,
params: ClearTaggingJobsParams,
) -> Result<usize, String> {
let conn = db.get().map_err(|e| e.to_string())?;
let requested_folder_ids = params.folder_ids.unwrap_or_default();
let (n, folder_ids): (usize, Vec<i64>) = match (params.folder_id, requested_folder_ids.is_empty()) {
(Some(id), _) => (
db::clear_tagging_jobs(&conn, Some(id)).map_err(|e| e.to_string())?,
vec![id],
),
(None, false) => {
let mut total = 0usize;
for &folder_id in &requested_folder_ids {
total += db::clear_tagging_jobs(&conn, Some(folder_id)).map_err(|e| e.to_string())?;
}
(total, requested_folder_ids)
}
(None, true) => (
db::clear_tagging_jobs(&conn, None).map_err(|e| e.to_string())?,
db::get_folders(&conn)
.map_err(|e| e.to_string())?
.into_iter()
.map(|f| f.id)
.collect(),
),
};
drop(conn);
indexer::emit_folder_job_progress(&app, db.inner(), &folder_ids, true);
Ok(n)
}
#[tauri::command]
pub async fn get_image_tags(
db: State<'_, DbState>,
params: GetImageTagsParams,
) -> Result<Vec<ImageTag>, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::get_image_tags(&conn, params.image_id).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn add_user_tag(
db: State<'_, DbState>,
params: AddUserTagParams,
) -> Result<ImageTag, String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::add_user_tag(&conn, params.image_id, &params.tag).map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn remove_tag(db: State<'_, DbState>, params: RemoveTagParams) -> Result<(), String> {
let conn = db.get().map_err(|e| e.to_string())?;
db::remove_tag(&conn, params.tag_id).map_err(|e| e.to_string())
}
+701 -9
View File
@@ -61,6 +61,10 @@ pub struct ImageRecord {
pub caption_model: Option<String>, pub caption_model: Option<String>,
pub caption_updated_at: Option<String>, pub caption_updated_at: Option<String>,
pub caption_error: Option<String>, pub caption_error: Option<String>,
pub ai_rating: Option<String>,
pub ai_tagger_model: Option<String>,
pub ai_tagged_at: Option<String>,
pub ai_tagger_error: Option<String>,
} }
#[allow(dead_code)] #[allow(dead_code)]
@@ -100,6 +104,32 @@ pub struct CaptionJob {
pub path: String, pub path: String,
} }
#[derive(Debug, Clone)]
pub struct TaggingJob {
pub image_id: i64,
pub folder_id: i64,
pub path: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ImageTag {
pub id: i64,
pub image_id: i64,
pub tag: String,
pub source: String,
pub ai_model: Option<String>,
pub confidence: Option<f64>,
pub created_at: String,
}
#[derive(Debug, Clone, Serialize)]
pub struct ExploreTagEntry {
pub tag: String,
pub count: i64,
pub representative_image_id: i64,
pub thumbnail_path: Option<String>,
}
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct IndexedMediaEntry { pub struct IndexedMediaEntry {
pub id: i64, pub id: i64,
@@ -120,6 +150,9 @@ pub struct FolderJobProgress {
pub caption_pending: i64, pub caption_pending: i64,
pub caption_ready: i64, pub caption_ready: i64,
pub caption_failed: i64, pub caption_failed: i64,
pub tagging_pending: i64,
pub tagging_ready: i64,
pub tagging_failed: i64,
} }
pub fn create_pool(db_path: &Path) -> Result<DbPool> { pub fn create_pool(db_path: &Path) -> Result<DbPool> {
@@ -195,6 +228,26 @@ pub fn migrate(conn: &Connection) -> Result<()> {
updated_at TEXT NOT NULL DEFAULT (datetime('now')) updated_at TEXT NOT NULL DEFAULT (datetime('now'))
); );
CREATE TABLE IF NOT EXISTS tagging_jobs (
image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE,
status TEXT NOT NULL DEFAULT 'pending',
attempts INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS image_tags (
id INTEGER PRIMARY KEY AUTOINCREMENT,
image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
tag TEXT NOT NULL,
source TEXT NOT NULL DEFAULT 'user',
ai_model TEXT,
confidence REAL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(image_id, tag)
);
CREATE TABLE IF NOT EXISTS tag_cloud_cache ( CREATE TABLE IF NOT EXISTS tag_cloud_cache (
folder_scope TEXT PRIMARY KEY, folder_scope TEXT PRIMARY KEY,
image_ids_hash INTEGER NOT NULL, image_ids_hash INTEGER NOT NULL,
@@ -202,12 +255,22 @@ pub fn migrate(conn: &Connection) -> Result<()> {
created_at TEXT NOT NULL DEFAULT (datetime('now')) created_at TEXT NOT NULL DEFAULT (datetime('now'))
); );
CREATE TABLE IF NOT EXISTS duplicate_scan_cache (
folder_scope TEXT PRIMARY KEY,
scanned_at INTEGER NOT NULL,
groups_json TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_images_folder_id ON images(folder_id); CREATE INDEX IF NOT EXISTS idx_images_folder_id ON images(folder_id);
CREATE INDEX IF NOT EXISTS idx_images_modified_at ON images(modified_at); CREATE INDEX IF NOT EXISTS idx_images_modified_at ON images(modified_at);
CREATE INDEX IF NOT EXISTS idx_embedding_jobs_status ON embedding_jobs(status); CREATE INDEX IF NOT EXISTS idx_embedding_jobs_status ON embedding_jobs(status);
CREATE INDEX IF NOT EXISTS idx_thumbnail_jobs_status ON thumbnail_jobs(status); CREATE INDEX IF NOT EXISTS idx_thumbnail_jobs_status ON thumbnail_jobs(status);
CREATE INDEX IF NOT EXISTS idx_metadata_jobs_status ON metadata_jobs(status); CREATE INDEX IF NOT EXISTS idx_metadata_jobs_status ON metadata_jobs(status);
CREATE INDEX IF NOT EXISTS idx_caption_jobs_status ON caption_jobs(status); CREATE INDEX IF NOT EXISTS idx_caption_jobs_status ON caption_jobs(status);
CREATE INDEX IF NOT EXISTS idx_tagging_jobs_status ON tagging_jobs(status);
CREATE INDEX IF NOT EXISTS idx_image_tags_image_id ON image_tags(image_id);
CREATE INDEX IF NOT EXISTS idx_image_tags_source ON image_tags(source);
CREATE INDEX IF NOT EXISTS idx_image_tags_tag ON image_tags(tag);
", ",
)?; )?;
@@ -237,6 +300,10 @@ pub fn migrate(conn: &Connection) -> Result<()> {
ensure_column(conn, "images", "caption_model", "TEXT")?; ensure_column(conn, "images", "caption_model", "TEXT")?;
ensure_column(conn, "images", "caption_updated_at", "TEXT")?; ensure_column(conn, "images", "caption_updated_at", "TEXT")?;
ensure_column(conn, "images", "caption_error", "TEXT")?; ensure_column(conn, "images", "caption_error", "TEXT")?;
ensure_column(conn, "images", "ai_rating", "TEXT")?;
ensure_column(conn, "images", "ai_tagger_model", "TEXT")?;
ensure_column(conn, "images", "ai_tagged_at", "TEXT")?;
ensure_column(conn, "images", "ai_tagger_error", "TEXT")?;
vector::migrate(conn)?; vector::migrate(conn)?;
Ok(()) Ok(())
@@ -257,8 +324,8 @@ pub fn insert_folder(conn: &Connection, path: &str, name: &str) -> Result<i64> {
pub fn upsert_image(conn: &Connection, img: &ImageRecord) -> Result<i64> { pub fn upsert_image(conn: &Connection, img: &ImageRecord) -> Result<i64> {
let id = conn.query_row( let id = conn.query_row(
"INSERT INTO images (folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type, media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error, favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error, generated_caption, caption_model, caption_updated_at, caption_error) "INSERT INTO images (folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type, media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error, favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error, generated_caption, caption_model, caption_updated_at, caption_error, ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19, ?20, ?21, ?22, ?23, ?24, ?25, ?26) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19, ?20, ?21, ?22, ?23, ?24, ?25, ?26, ?27, ?28, ?29, ?30)
ON CONFLICT(path) DO UPDATE SET ON CONFLICT(path) DO UPDATE SET
folder_id = excluded.folder_id, folder_id = excluded.folder_id,
filename = excluded.filename, filename = excluded.filename,
@@ -311,6 +378,10 @@ pub fn upsert_image(conn: &Connection, img: &ImageRecord) -> Result<i64> {
img.caption_model, img.caption_model,
img.caption_updated_at, img.caption_updated_at,
img.caption_error, img.caption_error,
img.ai_rating,
img.ai_tagger_model,
img.ai_tagged_at,
img.ai_tagger_error,
], ],
|row| row.get(0), |row| row.get(0),
)?; )?;
@@ -433,6 +504,13 @@ pub fn reset_inflight_jobs(conn: &Connection) -> Result<()> {
"UPDATE caption_jobs SET status = 'pending' WHERE status = 'processing'", "UPDATE caption_jobs SET status = 'pending' WHERE status = 'processing'",
[], [],
)?; )?;
conn.execute(
"UPDATE tagging_jobs SET status = 'pending' WHERE status = 'processing'",
[],
)?;
// Delete any rows that were cancelled before the previous shutdown so
// they don't silently linger in the DB across restarts.
conn.execute("DELETE FROM tagging_jobs WHERE status = 'cancelled'", [])?;
Ok(()) Ok(())
} }
@@ -493,6 +571,110 @@ pub fn requeue_caption_jobs(conn: &Connection, image_ids: &[i64]) -> Result<()>
Ok(()) Ok(())
} }
pub fn requeue_processing_caption_jobs_for_folder(
conn: &Connection,
folder_id: i64,
) -> Result<usize> {
conn.execute(
"UPDATE caption_jobs
SET status = 'pending', updated_at = datetime('now')
WHERE status = 'processing'
AND image_id IN (
SELECT id FROM images WHERE folder_id = ?1
)",
[folder_id],
)
.map_err(Into::into)
}
pub fn clear_caption_jobs(conn: &Connection, folder_id: Option<i64>) -> Result<usize> {
match folder_id {
Some(folder_id) => {
let deleted = conn.execute(
"DELETE FROM caption_jobs
WHERE image_id IN (
SELECT id FROM images WHERE folder_id = ?1
)",
[folder_id],
)?;
conn.execute(
"UPDATE images
SET caption_error = NULL
WHERE folder_id = ?1
AND generated_caption IS NULL",
[folder_id],
)?;
Ok(deleted)
}
None => {
let deleted = conn.execute("DELETE FROM caption_jobs", [])?;
conn.execute(
"UPDATE images
SET caption_error = NULL
WHERE generated_caption IS NULL",
[],
)?;
Ok(deleted)
}
}
}
pub fn reset_generated_captions(conn: &Connection, folder_id: Option<i64>) -> Result<usize> {
let image_ids = match folder_id {
Some(folder_id) => {
let mut stmt = conn.prepare(
"SELECT id FROM images WHERE folder_id = ?1 AND generated_caption IS NOT NULL",
)?;
let rows = stmt.query_map([folder_id], |row| row.get::<_, i64>(0))?;
rows.collect::<rusqlite::Result<Vec<_>>>()?
}
None => {
let mut stmt =
conn.prepare("SELECT id FROM images WHERE generated_caption IS NOT NULL")?;
let rows = stmt.query_map([], |row| row.get::<_, i64>(0))?;
rows.collect::<rusqlite::Result<Vec<_>>>()?
}
};
let tx = conn.unchecked_transaction()?;
for image_id in &image_ids {
vector::delete_caption_embedding(&tx, *image_id)?;
}
match folder_id {
Some(folder_id) => {
tx.execute(
"UPDATE images
SET generated_caption = NULL,
caption_model = NULL,
caption_updated_at = NULL,
caption_error = NULL
WHERE folder_id = ?1",
[folder_id],
)?;
tx.execute(
"DELETE FROM caption_jobs
WHERE image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[folder_id],
)?;
}
None => {
tx.execute(
"UPDATE images
SET generated_caption = NULL,
caption_model = NULL,
caption_updated_at = NULL,
caption_error = NULL",
[],
)?;
tx.execute("DELETE FROM caption_jobs", [])?;
}
}
tx.commit()?;
Ok(image_ids.len())
}
pub fn enqueue_missing_caption_jobs_for_folder(conn: &Connection, folder_id: i64) -> Result<usize> { pub fn enqueue_missing_caption_jobs_for_folder(conn: &Connection, folder_id: i64) -> Result<usize> {
let inserted = conn.execute( let inserted = conn.execute(
"INSERT INTO caption_jobs (image_id, status, attempts, last_error, created_at, updated_at) "INSERT INTO caption_jobs (image_id, status, attempts, last_error, created_at, updated_at)
@@ -791,9 +973,36 @@ pub fn get_folder_job_progress(conn: &Connection, folder_id: i64) -> Result<Fold
)?; )?;
let caption_failed = conn.query_row( let caption_failed = conn.query_row(
"SELECT COUNT(*)
FROM caption_jobs j
JOIN images i ON i.id = j.image_id
WHERE i.folder_id = ?1 AND j.status = 'failed'",
[folder_id],
|row| row.get(0),
)?;
let tagging_pending = conn.query_row(
"SELECT COUNT(*)
FROM tagging_jobs j
JOIN images i ON i.id = j.image_id
WHERE i.folder_id = ?1 AND j.status IN ('pending', 'processing')",
[folder_id],
|row| row.get(0),
)?;
let tagging_ready = conn.query_row(
"SELECT COUNT(*) "SELECT COUNT(*)
FROM images FROM images
WHERE folder_id = ?1 AND caption_error IS NOT NULL", WHERE folder_id = ?1 AND ai_tagged_at IS NOT NULL",
[folder_id],
|row| row.get(0),
)?;
let tagging_failed = conn.query_row(
"SELECT COUNT(*)
FROM tagging_jobs j
JOIN images i ON i.id = j.image_id
WHERE i.folder_id = ?1 AND j.status = 'failed'",
[folder_id], [folder_id],
|row| row.get(0), |row| row.get(0),
)?; )?;
@@ -808,6 +1017,9 @@ pub fn get_folder_job_progress(conn: &Connection, folder_id: i64) -> Result<Fold
caption_pending, caption_pending,
caption_ready, caption_ready,
caption_failed, caption_failed,
tagging_pending,
tagging_ready,
tagging_failed,
}) })
} }
@@ -1046,7 +1258,8 @@ pub fn update_image_details(
"SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type, "SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type,
media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error, media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error,
favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error, favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error,
generated_caption, caption_model, caption_updated_at, caption_error generated_caption, caption_model, caption_updated_at, caption_error,
ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error
FROM images FROM images
WHERE id = ?1", WHERE id = ?1",
[image_id], [image_id],
@@ -1060,7 +1273,8 @@ pub fn get_image_by_id(conn: &Connection, image_id: i64) -> Result<ImageRecord>
"SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type, "SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type,
media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error, media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error,
favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error, favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error,
generated_caption, caption_model, caption_updated_at, caption_error generated_caption, caption_model, caption_updated_at, caption_error,
ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error
FROM images FROM images
WHERE id = ?1", WHERE id = ?1",
[image_id], [image_id],
@@ -1102,6 +1316,7 @@ pub fn get_images(
search: Option<&str>, search: Option<&str>,
media_kind: Option<&str>, media_kind: Option<&str>,
favorites_only: bool, favorites_only: bool,
rating_min: i64,
embedding_failed_only: bool, embedding_failed_only: bool,
sort: &str, sort: &str,
offset: i64, offset: i64,
@@ -1114,6 +1329,8 @@ pub fn get_images(
"date_desc" => "modified_at DESC NULLS LAST", "date_desc" => "modified_at DESC NULLS LAST",
"size_asc" => "file_size ASC", "size_asc" => "file_size ASC",
"size_desc" => "file_size DESC", "size_desc" => "file_size DESC",
"rating_asc" => "rating ASC, modified_at DESC NULLS LAST",
"rating_desc" => "rating DESC, modified_at DESC NULLS LAST",
"duration_asc" => "duration_ms ASC NULLS LAST", "duration_asc" => "duration_ms ASC NULLS LAST",
"duration_desc" => "duration_ms DESC NULLS LAST", "duration_desc" => "duration_ms DESC NULLS LAST",
_ => "modified_at DESC NULLS LAST", _ => "modified_at DESC NULLS LAST",
@@ -1126,15 +1343,17 @@ pub fn get_images(
"SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type, "SELECT id, folder_id, path, filename, thumbnail_path, width, height, file_size, created_at, modified_at, mime_type,
media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error, media_kind, duration_ms, video_codec, audio_codec, metadata_updated_at, metadata_error,
favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error, favorite, rating, embedding_status, embedding_model, embedding_updated_at, embedding_error,
generated_caption, caption_model, caption_updated_at, caption_error generated_caption, caption_model, caption_updated_at, caption_error,
ai_rating, ai_tagger_model, ai_tagged_at, ai_tagger_error
FROM images FROM images
WHERE (?1 IS NULL OR folder_id = ?1) WHERE (?1 IS NULL OR folder_id = ?1)
AND (?2 IS NULL OR filename LIKE ?2) AND (?2 IS NULL OR filename LIKE ?2)
AND (?3 IS NULL OR media_kind = ?3) AND (?3 IS NULL OR media_kind = ?3)
AND (?4 = 0 OR favorite = 1) AND (?4 = 0 OR favorite = 1)
AND (?5 = 0 OR embedding_status = 'failed') AND rating >= ?5
AND (?6 = 0 OR embedding_status = 'failed')
ORDER BY {} ORDER BY {}
LIMIT ?6 OFFSET ?7", LIMIT ?7 OFFSET ?8",
order order
); );
let mut stmt = conn.prepare(&sql)?; let mut stmt = conn.prepare(&sql)?;
@@ -1144,6 +1363,7 @@ pub fn get_images(
search_pattern, search_pattern,
media_kind, media_kind,
favorites_flag, favorites_flag,
rating_min,
embedding_failed_flag, embedding_failed_flag,
limit, limit,
offset offset
@@ -1159,6 +1379,7 @@ pub fn count_images(
search: Option<&str>, search: Option<&str>,
media_kind: Option<&str>, media_kind: Option<&str>,
favorites_only: bool, favorites_only: bool,
rating_min: i64,
embedding_failed_only: bool, embedding_failed_only: bool,
) -> Result<i64> { ) -> Result<i64> {
let search_pattern = search.map(|value| format!("%{}%", value)); let search_pattern = search.map(|value| format!("%{}%", value));
@@ -1171,12 +1392,14 @@ pub fn count_images(
AND (?2 IS NULL OR filename LIKE ?2) AND (?2 IS NULL OR filename LIKE ?2)
AND (?3 IS NULL OR media_kind = ?3) AND (?3 IS NULL OR media_kind = ?3)
AND (?4 = 0 OR favorite = 1) AND (?4 = 0 OR favorite = 1)
AND (?5 = 0 OR embedding_status = 'failed')", AND rating >= ?5
AND (?6 = 0 OR embedding_status = 'failed')",
params![ params![
folder_id, folder_id,
search_pattern, search_pattern,
media_kind, media_kind,
favorites_flag, favorites_flag,
rating_min,
embedding_failed_flag embedding_failed_flag
], ],
|row| row.get(0), |row| row.get(0),
@@ -1185,6 +1408,141 @@ pub fn count_images(
Ok(count) Ok(count)
} }
pub fn search_images_by_tag(
conn: &Connection,
query: &str,
folder_id: Option<i64>,
media_kind: Option<&str>,
favorites_only: bool,
rating_min: i64,
limit: usize,
) -> Result<Vec<ImageRecord>> {
let normalized_query = query.trim().to_ascii_lowercase();
if normalized_query.is_empty() {
return Ok(Vec::new());
}
let favorites_flag = i64::from(favorites_only);
let mut stmt = conn.prepare(
"SELECT DISTINCT i.id
FROM images i
JOIN image_tags t ON t.image_id = i.id
WHERE (?1 IS NULL OR i.folder_id = ?1)
AND (?2 IS NULL OR i.media_kind = ?2)
AND (?3 = 0 OR i.favorite = 1)
AND i.rating >= ?4
AND LOWER(TRIM(t.tag)) = ?5
ORDER BY i.rating DESC, i.modified_at DESC NULLS LAST, i.filename ASC
LIMIT ?6",
)?;
let image_ids = stmt
.query_map(
params![
folder_id,
media_kind,
favorites_flag,
rating_min,
normalized_query,
limit as i64
],
|row| row.get::<_, i64>(0),
)?
.collect::<rusqlite::Result<Vec<_>>>()?;
get_images_by_ids(conn, &image_ids)
}
pub fn search_tags_autocomplete(
conn: &Connection,
query: &str,
folder_id: Option<i64>,
limit: usize,
) -> Result<Vec<ExploreTagEntry>> {
let pattern = format!("%{}%", query.to_lowercase());
let mut stmt = conn.prepare(
"SELECT t.tag, COUNT(DISTINCT t.image_id) AS tag_count, MIN(t.image_id) AS representative_image_id
FROM image_tags t
JOIN images i ON i.id = t.image_id
WHERE (?1 IS NULL OR i.folder_id = ?1)
AND LOWER(t.tag) LIKE ?2
GROUP BY t.tag
ORDER BY tag_count DESC, t.tag ASC
LIMIT ?3",
)?;
let rows = stmt
.query_map(params![folder_id, pattern, limit as i64], |row| {
Ok(ExploreTagEntry {
tag: row.get(0)?,
count: row.get(1)?,
representative_image_id: row.get(2)?,
thumbnail_path: None, // skip per-suggestion thumbnail for speed
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub struct ImagePathRecord {
pub id: i64,
pub path: String,
pub thumbnail_path: Option<String>,
}
pub fn get_all_image_paths(
conn: &Connection,
folder_id: Option<i64>,
) -> Result<Vec<ImagePathRecord>> {
let mut stmt = conn.prepare(
"SELECT id, path, thumbnail_path FROM images WHERE (?1 IS NULL OR folder_id = ?1) ORDER BY id",
)?;
let rows = stmt
.query_map(params![folder_id], |row| {
Ok(ImagePathRecord {
id: row.get(0)?,
path: row.get(1)?,
thumbnail_path: row.get(2)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub fn get_explore_tags(
conn: &Connection,
folder_id: Option<i64>,
limit: usize,
) -> Result<Vec<ExploreTagEntry>> {
let mut stmt = conn.prepare(
"SELECT t.tag, COUNT(DISTINCT t.image_id) AS tag_count, MIN(t.image_id) AS representative_image_id
FROM image_tags t
JOIN images i ON i.id = t.image_id
WHERE (?1 IS NULL OR i.folder_id = ?1)
GROUP BY t.tag
HAVING COUNT(DISTINCT t.image_id) >= 2
ORDER BY tag_count DESC, t.tag ASC
LIMIT ?2",
)?;
let rows = stmt
.query_map(params![folder_id, limit as i64], |row| {
let representative_image_id = row.get::<_, i64>(2)?;
let thumbnail_path = get_image_by_id(conn, representative_image_id)
.ok()
.and_then(|image| image.thumbnail_path);
Ok(ExploreTagEntry {
tag: row.get(0)?,
count: row.get(1)?,
representative_image_id,
thumbnail_path,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub fn get_failed_embedding_images( pub fn get_failed_embedding_images(
conn: &Connection, conn: &Connection,
folder_id: i64, folder_id: i64,
@@ -1240,6 +1598,299 @@ pub fn mark_caption_failed(conn: &Connection, image_id: i64, error: &str) -> Res
Ok(()) Ok(())
} }
// ---------------------------------------------------------------------------
// Tagging jobs
// ---------------------------------------------------------------------------
pub fn enqueue_tagging_job(conn: &Connection, image_id: i64) -> Result<()> {
conn.execute(
"INSERT INTO tagging_jobs (image_id, status, attempts, last_error, created_at, updated_at)
VALUES (?1, 'pending', 0, NULL, datetime('now'), datetime('now'))
ON CONFLICT(image_id) DO UPDATE SET
status = CASE WHEN status != 'processing' THEN 'pending' ELSE status END,
last_error = CASE WHEN status != 'processing' THEN NULL ELSE last_error END,
updated_at = datetime('now')",
[image_id],
)?;
Ok(())
}
pub fn claim_tagging_jobs(
conn: &mut Connection,
paused_folder_ids: &std::collections::HashSet<i64>,
limit: usize,
) -> Result<Vec<TaggingJob>> {
let tx = conn.transaction()?;
let candidates = get_pending_tagging_jobs_excluding(&tx, paused_folder_ids, limit * 2)?;
let mut claimed = Vec::new();
for job in candidates {
let updated = tx.execute(
"UPDATE tagging_jobs
SET status = 'processing', attempts = attempts + 1, updated_at = datetime('now')
WHERE image_id = ?1 AND status = 'pending'",
[job.image_id],
)?;
if updated > 0 {
claimed.push(job);
if claimed.len() >= limit {
break;
}
}
}
tx.commit()?;
Ok(claimed)
}
fn get_pending_tagging_jobs_excluding(
conn: &Connection,
paused_folder_ids: &std::collections::HashSet<i64>,
limit: usize,
) -> Result<Vec<TaggingJob>> {
let mut stmt = conn.prepare(
"SELECT j.image_id, i.folder_id, i.path
FROM tagging_jobs j
JOIN images i ON i.id = j.image_id
WHERE j.status = 'pending'
ORDER BY j.created_at ASC
LIMIT ?1",
)?;
let rows = stmt
.query_map([limit as i64], |row| {
Ok(TaggingJob {
image_id: row.get(0)?,
folder_id: row.get(1)?,
path: row.get(2)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows
.into_iter()
.filter(|job| !paused_folder_ids.contains(&job.folder_id))
.collect())
}
pub fn update_ai_tags(
conn: &Connection,
image_id: i64,
tags: &[(String, f64)],
rating: &str,
model: &str,
) -> Result<()> {
// NOTE: callers are responsible for wrapping this in a transaction.
// Do NOT open a nested transaction here — rusqlite/SQLite do not support
// nested transactions and will error with "cannot start a transaction
// within a transaction".
// Remove previous AI tags for this image before inserting fresh ones
conn.execute(
"DELETE FROM image_tags WHERE image_id = ?1 AND source = 'ai'",
[image_id],
)?;
let mut stmt = conn.prepare(
"INSERT INTO image_tags (image_id, tag, source, ai_model, confidence, created_at)
VALUES (?1, ?2, 'ai', ?3, ?4, datetime('now'))
ON CONFLICT(image_id, tag) DO UPDATE SET
source = 'ai',
ai_model = excluded.ai_model,
confidence = excluded.confidence",
)?;
for (tag, confidence) in tags {
stmt.execute(params![image_id, tag, model, confidence])?;
}
drop(stmt);
conn.execute(
"UPDATE images
SET ai_rating = ?2,
ai_tagger_model = ?3,
ai_tagged_at = datetime('now'),
ai_tagger_error = NULL
WHERE id = ?1",
params![image_id, rating, model],
)?;
conn.execute("DELETE FROM tagging_jobs WHERE image_id = ?1", [image_id])?;
Ok(())
}
pub fn mark_tagging_failed(conn: &Connection, image_id: i64, error: &str) -> Result<()> {
conn.execute(
"UPDATE images SET ai_tagger_error = ?2 WHERE id = ?1",
params![image_id, error],
)?;
conn.execute(
"UPDATE tagging_jobs
SET status = 'failed', last_error = ?2, updated_at = datetime('now')
WHERE image_id = ?1",
params![image_id, error],
)?;
Ok(())
}
pub fn clear_tagging_jobs(conn: &Connection, folder_id: Option<i64>) -> Result<usize> {
// Rows currently being processed by the worker must not be deleted mid-flight
// — the worker holds a reference and will write results back after inference.
// Mark them 'cancelled' so the worker discards the result instead of saving it,
// then delete every non-processing row. On the next poll the worker will find
// no pending work and the queue will appear empty.
let deleted = match folder_id {
Some(fid) => {
conn.execute(
"UPDATE tagging_jobs
SET status = 'cancelled', last_error = NULL, updated_at = datetime('now')
WHERE status = 'processing'
AND image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[fid],
)?;
let n = conn.execute(
"DELETE FROM tagging_jobs
WHERE status != 'processing'
AND image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[fid],
)?;
conn.execute(
"UPDATE images
SET ai_tagger_error = NULL
WHERE folder_id = ?1 AND ai_tagged_at IS NULL",
[fid],
)?;
n
}
None => {
conn.execute(
"UPDATE tagging_jobs
SET status = 'cancelled', last_error = NULL, updated_at = datetime('now')
WHERE status = 'processing'",
[],
)?;
let n = conn.execute("DELETE FROM tagging_jobs WHERE status != 'processing'", [])?;
conn.execute(
"UPDATE images SET ai_tagger_error = NULL WHERE ai_tagged_at IS NULL",
[],
)?;
n
}
};
Ok(deleted)
}
pub fn get_image_tags(conn: &Connection, image_id: i64) -> Result<Vec<ImageTag>> {
let mut stmt = conn.prepare(
"SELECT id, image_id, tag, source, ai_model, confidence, created_at
FROM image_tags
WHERE image_id = ?1
ORDER BY source DESC, confidence DESC NULLS LAST, tag ASC",
)?;
let rows = stmt
.query_map([image_id], |row| {
Ok(ImageTag {
id: row.get(0)?,
image_id: row.get(1)?,
tag: row.get(2)?,
source: row.get(3)?,
ai_model: row.get(4)?,
confidence: row.get(5)?,
created_at: row.get(6)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows)
}
pub fn add_user_tag(conn: &Connection, image_id: i64, tag: &str) -> Result<ImageTag> {
conn.execute(
"INSERT INTO image_tags (image_id, tag, source, ai_model, confidence, created_at)
VALUES (?1, ?2, 'user', NULL, NULL, datetime('now'))
ON CONFLICT(image_id, tag) DO NOTHING",
params![image_id, tag],
)?;
let row = conn.query_row(
"SELECT id, image_id, tag, source, ai_model, confidence, created_at
FROM image_tags WHERE image_id = ?1 AND tag = ?2",
params![image_id, tag],
|row| {
Ok(ImageTag {
id: row.get(0)?,
image_id: row.get(1)?,
tag: row.get(2)?,
source: row.get(3)?,
ai_model: row.get(4)?,
confidence: row.get(5)?,
created_at: row.get(6)?,
})
},
)?;
Ok(row)
}
pub fn remove_tag(conn: &Connection, tag_id: i64) -> Result<()> {
conn.execute("DELETE FROM image_tags WHERE id = ?1", [tag_id])?;
Ok(())
}
pub fn enqueue_missing_tagging_jobs_for_folder(conn: &Connection, folder_id: i64) -> Result<usize> {
let inserted = conn.execute(
"INSERT INTO tagging_jobs (image_id, status, attempts, last_error, created_at, updated_at)
SELECT id, 'pending', 0, NULL, datetime('now'), datetime('now')
FROM images
WHERE folder_id = ?1
AND media_kind = 'image'
AND ai_tagged_at IS NULL
ON CONFLICT(image_id) DO UPDATE SET
-- Only reset to 'pending' if the row is not currently being
-- processed or cancelled; leave 'processing' rows untouched so
-- the worker can complete them and clean up normally.
status = CASE WHEN status NOT IN ('processing', 'cancelled') THEN 'pending' ELSE status END,
last_error = CASE WHEN status != 'processing' THEN NULL ELSE last_error END,
updated_at = datetime('now')",
[folder_id],
)?;
conn.execute(
"UPDATE images
SET ai_tagger_error = NULL
WHERE folder_id = ?1
AND media_kind = 'image'
AND ai_tagged_at IS NULL",
[folder_id],
)?;
Ok(inserted)
}
pub fn requeue_processing_tagging_jobs_for_folder(conn: &Connection, folder_id: i64) -> Result<()> {
conn.execute(
"UPDATE tagging_jobs
SET status = 'pending', updated_at = datetime('now')
WHERE status = 'processing'
AND image_id IN (SELECT id FROM images WHERE folder_id = ?1)",
[folder_id],
)?;
Ok(())
}
/// Returns `true` when the job row for `image_id` currently has status = 'cancelled'.
/// Used by the worker to discard inference results for jobs that were cancelled
/// while inference was running.
pub fn is_tagging_job_cancelled(conn: &Connection, image_id: i64) -> Result<bool> {
let count: i64 = conn.query_row(
"SELECT COUNT(*) FROM tagging_jobs WHERE image_id = ?1 AND status = 'cancelled'",
[image_id],
|row| row.get(0),
)?;
Ok(count > 0)
}
pub fn requeue_tagging_jobs(conn: &Connection, image_ids: &[i64]) -> Result<()> {
for image_id in image_ids {
conn.execute(
"UPDATE tagging_jobs
SET status = 'pending', updated_at = datetime('now')
WHERE image_id = ?1 AND status = 'processing'",
[image_id],
)?;
}
Ok(())
}
pub fn suggest_tags_from_caption( pub fn suggest_tags_from_caption(
conn: &Connection, conn: &Connection,
image_id: i64, image_id: i64,
@@ -1332,6 +1983,10 @@ fn map_image_row(row: &Row<'_>) -> rusqlite::Result<ImageRecord> {
caption_model: row.get(24)?, caption_model: row.get(24)?,
caption_updated_at: row.get(25)?, caption_updated_at: row.get(25)?,
caption_error: row.get(26)?, caption_error: row.get(26)?,
ai_rating: row.get(27)?,
ai_tagger_model: row.get(28)?,
ai_tagged_at: row.get(29)?,
ai_tagger_error: row.get(30)?,
}) })
} }
@@ -1373,6 +2028,43 @@ pub fn set_tag_cloud_cache(
Ok(()) Ok(())
} }
/// Returns (groups_json, scanned_at_unix) for the given folder scope, if present.
pub fn get_duplicate_scan_cache(
conn: &Connection,
folder_scope: &str,
) -> Result<Option<(String, i64)>> {
match conn.query_row(
"SELECT groups_json, scanned_at FROM duplicate_scan_cache WHERE folder_scope = ?1",
params![folder_scope],
|row| Ok((row.get::<_, String>(0)?, row.get::<_, i64>(1)?)),
) {
Ok(row) => Ok(Some(row)),
Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
Err(e) => Err(e.into()),
}
}
/// Upserts the duplicate scan cache for the given scope.
pub fn set_duplicate_scan_cache(
conn: &Connection,
folder_scope: &str,
groups_json: &str,
) -> Result<()> {
let scanned_at = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
.unwrap_or(0);
conn.execute(
"INSERT INTO duplicate_scan_cache (folder_scope, scanned_at, groups_json)
VALUES (?1, ?2, ?3)
ON CONFLICT(folder_scope) DO UPDATE SET
scanned_at = excluded.scanned_at,
groups_json = excluded.groups_json",
params![folder_scope, scanned_at, groups_json],
)?;
Ok(())
}
fn ensure_column(conn: &Connection, table: &str, column: &str, definition: &str) -> Result<()> { fn ensure_column(conn: &Connection, table: &str, column: &str, definition: &str) -> Result<()> {
let mut stmt = conn.prepare(&format!("PRAGMA table_info({})", table))?; let mut stmt = conn.prepare(&format!("PRAGMA table_info({})", table))?;
let mut rows = stmt.query([])?; let mut rows = stmt.query([])?;
+45
View File
@@ -74,6 +74,51 @@ impl ClipImageEmbedder {
Ok(self.embed_images(&[path.to_path_buf()])?.remove(0)) Ok(self.embed_images(&[path.to_path_buf()])?.remove(0))
} }
/// Embed a cropped region of an image without writing a temp file to disk.
/// `crop_x`, `crop_y`, `crop_w`, `crop_h` are normalized 0.01.0 coordinates.
pub fn embed_image_crop(
&self,
path: &Path,
crop_x: f32,
crop_y: f32,
crop_w: f32,
crop_h: f32,
) -> Result<Vec<f32>> {
let img = image::ImageReader::open(path)?
.with_guessed_format()?
.decode()?;
let img_w = img.width() as f32;
let img_h = img.height() as f32;
let x = ((crop_x * img_w) as u32).min(img.width().saturating_sub(1));
let y = ((crop_y * img_h) as u32).min(img.height().saturating_sub(1));
let w = ((crop_w * img_w) as u32).max(1).min(img.width() - x);
let h = ((crop_h * img_h) as u32).max(1).min(img.height() - y);
let cropped = img.crop_imm(x, y, w, h);
let resized = cropped.resize_to_fill(
self.image_size as u32,
self.image_size as u32,
image::imageops::FilterType::Triangle,
);
let raw = resized.to_rgb8().into_raw();
let tensor = candle_core::Tensor::from_vec(
raw,
(self.image_size, self.image_size, 3),
&candle_core::Device::Cpu,
)?
.permute((2, 0, 1))?
.to_dtype(candle_core::DType::F32)?
.affine(2.0 / 255.0, -1.0)?;
let batch = tensor.unsqueeze(0)?.to_device(&self.device)?;
let features = self.model.get_image_features(&batch)?;
let normalized = candle_transformers::models::clip::div_l2_norm(&features)?;
Ok(normalized.get(0)?.flatten_all()?.to_vec1::<f32>()?)
}
pub fn embed_images(&self, paths: &[PathBuf]) -> Result<Vec<Vec<f32>>> { pub fn embed_images(&self, paths: &[PathBuf]) -> Result<Vec<Vec<f32>>> {
let images = load_images(paths, self.image_size)?.to_device(&self.device)?; let images = load_images(paths, self.image_size)?.to_device(&self.device)?;
let features = self.model.get_image_features(&images)?; let features = self.model.get_image_features(&images)?;
+129
View File
@@ -0,0 +1,129 @@
use crate::vector;
use anyhow::Result;
use hnsw_rs::prelude::{DistCosine, Hnsw, Neighbour};
use rusqlite::Connection;
use std::collections::HashMap;
use std::sync::{OnceLock, RwLock};
const HNSW_MAX_CONNECTIONS: usize = 24;
const HNSW_EF_CONSTRUCTION: usize = 300;
const HNSW_EF_SEARCH: usize = 96;
struct CachedHnswIndex {
revision: String,
image_ids_by_external: Vec<i64>,
external_by_image_id: HashMap<i64, usize>,
hnsw: Hnsw<'static, f32, DistCosine>,
}
static IMAGE_HNSW_INDEX: OnceLock<RwLock<Option<CachedHnswIndex>>> = OnceLock::new();
fn cache() -> &'static RwLock<Option<CachedHnswIndex>> {
IMAGE_HNSW_INDEX.get_or_init(|| RwLock::new(None))
}
fn build_index(conn: &Connection) -> Result<CachedHnswIndex> {
let embeddings = vector::get_all_image_embeddings_with_ids(conn, None)?;
let max_elements = embeddings.len().max(1);
let max_layer = 16.min((max_elements as f32).ln().trunc() as usize).max(1);
let mut hnsw = Hnsw::<f32, DistCosine>::new(
HNSW_MAX_CONNECTIONS,
max_elements,
max_layer,
HNSW_EF_CONSTRUCTION,
DistCosine {},
);
let image_ids_by_external = embeddings
.iter()
.map(|(image_id, _)| *image_id)
.collect::<Vec<_>>();
let external_by_image_id = image_ids_by_external
.iter()
.enumerate()
.map(|(external_id, image_id)| (*image_id, external_id))
.collect::<HashMap<_, _>>();
let data_with_id = embeddings
.iter()
.enumerate()
.map(|(external_id, (_, embedding))| (embedding, external_id))
.collect::<Vec<_>>();
hnsw.parallel_insert(&data_with_id);
hnsw.set_searching_mode(true);
Ok(CachedHnswIndex {
revision: vector::get_embedding_revision(conn)?,
image_ids_by_external,
external_by_image_id,
hnsw,
})
}
fn ensure_index(conn: &Connection) -> Result<()> {
let revision = vector::get_embedding_revision(conn)?;
{
let guard = cache().read().expect("hnsw cache poisoned");
if guard.as_ref().map(|cached| cached.revision.as_str()) == Some(revision.as_str()) {
return Ok(());
}
}
let next = build_index(conn)?;
let mut guard = cache().write().expect("hnsw cache poisoned");
*guard = Some(next);
Ok(())
}
pub fn find_similar_image_matches(
conn: &Connection,
image_id: i64,
folder_id: Option<i64>,
threshold: f32,
offset: usize,
limit: usize,
) -> Result<Vec<(i64, f32)>> {
ensure_index(conn)?;
let query_embedding = match vector::get_image_embedding(conn, image_id)? {
Some(embedding) => embedding,
None => return Ok(Vec::new()),
};
let guard = cache().read().expect("hnsw cache poisoned");
let Some(cached) = guard.as_ref() else {
return Ok(Vec::new());
};
let knbn = (offset + limit).max(limit).saturating_add(32);
let neighbours: Vec<Neighbour> = if let Some(folder_id) = folder_id {
let mut allowed_ids = vector::get_all_image_embeddings_with_ids(conn, Some(folder_id))?
.into_iter()
.filter_map(|(allowed_image_id, _)| {
cached.external_by_image_id.get(&allowed_image_id).copied()
})
.collect::<Vec<_>>();
allowed_ids.sort_unstable();
cached
.hnsw
.search_filter(&query_embedding, knbn, HNSW_EF_SEARCH, Some(&allowed_ids))
} else {
cached.hnsw.search(&query_embedding, knbn, HNSW_EF_SEARCH)
};
let matches = neighbours
.into_iter()
.filter_map(|neighbour| {
let image_id_match = cached.image_ids_by_external.get(neighbour.d_id).copied()?;
if image_id_match == image_id || neighbour.distance > threshold {
return None;
}
Some((image_id_match, neighbour.distance))
})
.skip(offset)
.take(limit)
.collect::<Vec<_>>();
Ok(matches)
}
+168 -1
View File
@@ -3,6 +3,7 @@ use crate::db::{self, DbPool, EmbeddingJob, FolderJobProgress, ImageRecord, Inde
use crate::embedder::{embedding_source_path, ClipImageEmbedder}; use crate::embedder::{embedding_source_path, ClipImageEmbedder};
use crate::media::{probe_video_metadata, MediaTools}; use crate::media::{probe_video_metadata, MediaTools};
use crate::storage::{detect_storage_profile, RuntimeAdaptiveProfile, StorageProfile}; use crate::storage::{detect_storage_profile, RuntimeAdaptiveProfile, StorageProfile};
use crate::tagger::{self, WdTagger};
use crate::thumbnail; use crate::thumbnail;
use crate::vector; use crate::vector;
use anyhow::Result; use anyhow::Result;
@@ -35,6 +36,7 @@ struct PausedWorkerFolders {
metadata: HashSet<i64>, metadata: HashSet<i64>,
embedding: HashSet<i64>, embedding: HashSet<i64>,
caption: HashSet<i64>, caption: HashSet<i64>,
tagging: HashSet<i64>,
} }
#[derive(Clone, Copy)] #[derive(Clone, Copy)]
@@ -43,6 +45,7 @@ pub struct FolderWorkerPausedState {
pub metadata: bool, pub metadata: bool,
pub embedding: bool, pub embedding: bool,
pub caption: bool, pub caption: bool,
pub tagging: bool,
} }
pub fn set_worker_paused(worker: &str, folder_id: i64, paused: bool) { pub fn set_worker_paused(worker: &str, folder_id: i64, paused: bool) {
@@ -55,6 +58,7 @@ pub fn set_worker_paused(worker: &str, folder_id: i64, paused: bool) {
"metadata" => Some(&mut paused_folders.metadata), "metadata" => Some(&mut paused_folders.metadata),
"embedding" => Some(&mut paused_folders.embedding), "embedding" => Some(&mut paused_folders.embedding),
"caption" => Some(&mut paused_folders.caption), "caption" => Some(&mut paused_folders.caption),
"tagging" => Some(&mut paused_folders.tagging),
_ => None, _ => None,
}; };
@@ -87,6 +91,7 @@ pub fn get_worker_paused_states(folder_ids: &[i64]) -> HashMap<i64, FolderWorker
metadata: paused_folders.metadata.contains(&folder_id), metadata: paused_folders.metadata.contains(&folder_id),
embedding: paused_folders.embedding.contains(&folder_id), embedding: paused_folders.embedding.contains(&folder_id),
caption: paused_folders.caption.contains(&folder_id), caption: paused_folders.caption.contains(&folder_id),
tagging: paused_folders.tagging.contains(&folder_id),
}, },
) )
}) })
@@ -105,6 +110,8 @@ fn paused_folder_ids(worker: &str) -> HashSet<i64> {
"thumbnail" => paused_folders.thumbnail.clone(), "thumbnail" => paused_folders.thumbnail.clone(),
"metadata" => paused_folders.metadata.clone(), "metadata" => paused_folders.metadata.clone(),
"embedding" => paused_folders.embedding.clone(), "embedding" => paused_folders.embedding.clone(),
"caption" => paused_folders.caption.clone(),
"tagging" => paused_folders.tagging.clone(),
_ => HashSet::new(), _ => HashSet::new(),
} }
} }
@@ -191,6 +198,12 @@ pub fn start_caption_worker(app: AppHandle, pool: DbPool, app_data_dir: PathBuf)
let mut captioner: Option<FlorenceCaptioner> = None; let mut captioner: Option<FlorenceCaptioner> = None;
println!("Caption worker started."); println!("Caption worker started.");
loop { loop {
// If the acceleration setting changed, drop the cached session so
// the next batch picks it up with the new execution provider.
if captioner::CAPTION_SESSION_DIRTY.swap(false, std::sync::atomic::Ordering::Relaxed) {
println!("Caption worker: acceleration setting changed — resetting session.");
captioner = None;
}
if let Err(error) = process_caption_batch(&app, &pool, &app_data_dir, &mut captioner) { if let Err(error) = process_caption_batch(&app, &pool, &app_data_dir, &mut captioner) {
eprintln!("Caption worker error: {}", error); eprintln!("Caption worker error: {}", error);
captioner = None; captioner = None;
@@ -200,6 +213,28 @@ pub fn start_caption_worker(app: AppHandle, pool: DbPool, app_data_dir: PathBuf)
}); });
} }
pub fn start_tagging_worker(app: AppHandle, pool: DbPool, app_data_dir: PathBuf) {
std::thread::spawn(move || {
let mut tagger_instance: Option<WdTagger> = None;
println!("Tagging worker started.");
loop {
// If the acceleration setting changed, drop the cached session so
// the next batch picks it up with the new execution provider.
if tagger::TAGGER_SESSION_DIRTY.swap(false, std::sync::atomic::Ordering::Relaxed) {
println!("Tagging worker: acceleration setting changed — resetting session.");
tagger_instance = None;
}
if let Err(error) =
process_tagging_batch(&app, &pool, &app_data_dir, &mut tagger_instance)
{
eprintln!("Tagging worker error: {}", error);
tagger_instance = None;
}
std::thread::sleep(std::time::Duration::from_millis(750));
}
});
}
fn do_index(app: AppHandle, pool: DbPool, folder_id: i64, folder_path: PathBuf) -> Result<()> { fn do_index(app: AppHandle, pool: DbPool, folder_id: i64, folder_path: PathBuf) -> Result<()> {
let existing_entries = { let existing_entries = {
let conn = pool.get()?; let conn = pool.get()?;
@@ -296,6 +331,7 @@ fn do_index(app: AppHandle, pool: DbPool, folder_id: i64, folder_path: PathBuf)
if !missing_ids.is_empty() { if !missing_ids.is_empty() {
db::delete_images_by_ids(&conn, &missing_ids)?; db::delete_images_by_ids(&conn, &missing_ids)?;
} }
let _ = db::backfill_embedding_jobs(&conn)?;
db::update_folder_count(&conn, folder_id)?; db::update_folder_count(&conn, folder_id)?;
} }
@@ -370,6 +406,10 @@ fn build_record(
caption_model: None, caption_model: None,
caption_updated_at: None, caption_updated_at: None,
caption_error: None, caption_error: None,
ai_rating: None,
ai_tagger_model: None,
ai_tagged_at: None,
ai_tagger_error: None,
}) })
} }
@@ -825,6 +865,133 @@ fn process_caption_batch(
Ok(()) Ok(())
} }
fn process_tagging_batch(
app: &AppHandle,
pool: &DbPool,
app_data_dir: &Path,
tagger_instance: &mut Option<WdTagger>,
) -> Result<()> {
if !tagger::tagger_model_status(app_data_dir).ready {
return Ok(());
}
let paused_folders = paused_folder_ids("tagging");
let batch_size = crate::tagger::tagger_batch_size(app_data_dir);
let jobs = with_db_write_lock(|| {
let mut conn = pool.get()?;
db::claim_tagging_jobs(&mut conn, &paused_folders, batch_size)
})?;
if jobs.is_empty() {
return Ok(());
}
if tagger_instance.is_none() {
match WdTagger::new(app_data_dir) {
Ok(model) => *tagger_instance = Some(model),
Err(error) => {
with_db_write_lock(|| {
let conn = pool.get()?;
db::requeue_tagging_jobs(
&conn,
&jobs.iter().map(|job| job.image_id).collect::<Vec<_>>(),
)
})?;
return Err(error);
}
}
}
let folder_ids = jobs.iter().map(|job| job.folder_id).collect::<HashSet<_>>();
emit_folder_job_progress(
app,
pool,
&folder_ids.iter().copied().collect::<Vec<_>>(),
false,
);
let tagger_ref = tagger_instance
.as_mut()
.expect("tagger should be initialized before tagging batch processing");
let tag_results = jobs
.iter()
.map(|job| {
(
job.clone(),
tagger_ref.run(Path::new(&job.path), tagger::DEFAULT_MAX_TAGS),
)
})
.collect::<Vec<_>>();
let updated_images = with_db_write_lock(|| {
let mut conn = pool.get()?;
let tx = conn.transaction()?;
let mut updated_images = Vec::with_capacity(tag_results.len());
for (job, tag_result) in &tag_results {
// If the job was cancelled while inference was running, discard
// the result and delete the row — don't save tags or mark failed.
if db::is_tagging_job_cancelled(&tx, job.image_id)? {
tx.execute(
"DELETE FROM tagging_jobs WHERE image_id = ?1",
[job.image_id],
)?;
continue;
}
match tag_result {
Ok(output) => {
let tag_pairs: Vec<(String, f64)> = output
.tags
.iter()
.map(|t| (t.tag.clone(), t.confidence as f64))
.collect();
db::update_ai_tags(
&tx,
job.image_id,
&tag_pairs,
&output.rating,
tagger::WD_TAGGER_MODEL_NAME,
)?;
}
Err(error) => {
db::mark_tagging_failed(&tx, job.image_id, &error.to_string())?;
}
}
updated_images.push(db::get_image_by_id(&tx, job.image_id)?);
}
tx.commit()?;
Ok(updated_images)
})
.or_else(|db_err| {
// The DB write failed. Try to requeue the claimed jobs so they aren't
// left stuck in 'processing' until the next app restart.
let image_ids: Vec<i64> = jobs.iter().map(|job| job.image_id).collect();
let _ = with_db_write_lock(|| {
let conn = pool.get()?;
db::requeue_tagging_jobs(&conn, &image_ids)
});
Err(db_err)
})?;
if !updated_images.is_empty() {
let folder_ids = updated_images
.iter()
.map(|image| image.folder_id)
.collect::<HashSet<_>>();
emit_media_updates(
app,
&MediaUpdateBatch {
images: updated_images,
},
);
emit_folder_job_progress(app, pool, &folder_ids.into_iter().collect::<Vec<_>>(), true);
}
Ok(())
}
fn active_indexing_folders() -> HashSet<i64> { fn active_indexing_folders() -> HashSet<i64> {
ACTIVE_INDEXING_FOLDERS ACTIVE_INDEXING_FOLDERS
.get_or_init(|| Mutex::new(HashSet::new())) .get_or_init(|| Mutex::new(HashSet::new()))
@@ -910,7 +1077,7 @@ fn emit_media_updates(app: &AppHandle, batch: &MediaUpdateBatch) {
let _ = app.emit("media-updated", batch); let _ = app.emit("media-updated", batch);
} }
fn emit_folder_job_progress(app: &AppHandle, pool: &DbPool, folder_ids: &[i64], force: bool) { pub fn emit_folder_job_progress(app: &AppHandle, pool: &DbPool, folder_ids: &[i64], force: bool) {
let mut unique_folder_ids = folder_ids.iter().copied().collect::<Vec<_>>(); let mut unique_folder_ids = folder_ids.iter().copied().collect::<Vec<_>>();
unique_folder_ids.sort_unstable(); unique_folder_ids.sort_unstable();
unique_folder_ids.dedup(); unique_folder_ids.dedup();
+11
View File
@@ -2,6 +2,7 @@ mod captioner;
mod commands; mod commands;
mod db; mod db;
mod embedder; mod embedder;
mod hnsw_index;
mod indexer; mod indexer;
mod media; mod media;
mod storage; mod storage;
@@ -86,9 +87,11 @@ pub fn run() {
commands::reindex_folder, commands::reindex_folder,
commands::update_image_details, commands::update_image_details,
commands::find_similar_images, commands::find_similar_images,
commands::find_similar_by_region,
commands::debug_similar_images, commands::debug_similar_images,
commands::retry_failed_embeddings, commands::retry_failed_embeddings,
commands::semantic_search_images, commands::semantic_search_images,
commands::search_images_by_tag,
commands::get_caption_model_status, commands::get_caption_model_status,
commands::get_caption_acceleration, commands::get_caption_acceleration,
commands::set_caption_acceleration, commands::set_caption_acceleration,
@@ -107,6 +110,8 @@ pub fn run() {
commands::set_worker_paused, commands::set_worker_paused,
commands::get_worker_states, commands::get_worker_states,
commands::get_tag_cloud, commands::get_tag_cloud,
commands::get_explore_tags,
commands::get_images_by_ids,
commands::get_failed_embedding_images, commands::get_failed_embedding_images,
commands::get_tagger_model_status, commands::get_tagger_model_status,
commands::get_tagger_acceleration, commands::get_tagger_acceleration,
@@ -114,6 +119,8 @@ pub fn run() {
commands::probe_tagger_runtime, commands::probe_tagger_runtime,
commands::get_tagger_threshold, commands::get_tagger_threshold,
commands::set_tagger_threshold, commands::set_tagger_threshold,
commands::get_tagger_batch_size,
commands::set_tagger_batch_size,
commands::prepare_tagger_model, commands::prepare_tagger_model,
commands::delete_tagger_model, commands::delete_tagger_model,
commands::queue_tagging_jobs, commands::queue_tagging_jobs,
@@ -121,6 +128,10 @@ pub fn run() {
commands::get_image_tags, commands::get_image_tags,
commands::add_user_tag, commands::add_user_tag,
commands::remove_tag, commands::remove_tag,
commands::search_tags_autocomplete,
commands::find_duplicates,
commands::load_duplicate_scan_cache,
commands::delete_images_from_disk,
]) ])
.run(tauri::generate_context!()) .run(tauri::generate_context!())
.expect("error while running tauri application"); .expect("error while running tauri application");
+648
View File
@@ -0,0 +1,648 @@
use anyhow::Result;
use hf_hub::{api::sync::Api, Repo, RepoType};
use image::{imageops::FilterType, DynamicImage, ImageReader};
use ort::ep;
use ort::session::{builder::GraphOptimizationLevel, Session};
use ort::value::Tensor;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Instant;
pub const WD_TAGGER_MODEL_ID: &str = "SmilingWolf/wd-swinv2-tagger-v3";
pub const WD_TAGGER_MODEL_NAME: &str = "wd-swinv2-tagger-v3";
const TAGGER_ACCELERATION_FILE: &str = "settings/tagger_acceleration.txt";
const TAGGER_THRESHOLD_FILE: &str = "settings/tagger_threshold.txt";
const TAGGER_BATCH_SIZE_FILE: &str = "settings/tagger_batch_size.txt";
// Files required on disk before the tagger can run. The ONNX runtime DLLs
// are shared with the captioner and live in the same `onnxruntime/` directory.
const TAGGER_REQUIRED_FILES: &[&str] = &[
"onnxruntime/onnxruntime.dll",
"onnxruntime/onnxruntime_providers_shared.dll",
"onnxruntime/DirectML.dll",
"model.onnx",
"selected_tags.csv",
];
// Tags in these Danbooru categories are kept in the output.
// Category 0 = general, category 4 = character.
// Category 9 = rating (explicit/questionable/sensitive/general) used for
// `ai_rating` but NOT emitted as individual tags.
const GENERAL_CATEGORY: u8 = 0;
const CHARACTER_CATEGORY: u8 = 4;
const RATING_CATEGORY: u8 = 9;
pub const DEFAULT_THRESHOLD: f32 = 0.35;
pub const DEFAULT_MAX_TAGS: usize = 30;
/// Set to `true` by `set_tagger_acceleration` so the tagging worker loop
/// knows to drop its cached `WdTagger` and reload with the new EP.
pub static TAGGER_SESSION_DIRTY: AtomicBool = AtomicBool::new(false);
// ---------------------------------------------------------------------------
// Settings types
// ---------------------------------------------------------------------------
#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum TaggerAcceleration {
Auto,
Cpu,
Directml,
}
impl TaggerAcceleration {
fn as_str(self) -> &'static str {
match self {
Self::Auto => "auto",
Self::Cpu => "cpu",
Self::Directml => "directml",
}
}
}
impl Default for TaggerAcceleration {
fn default() -> Self {
Self::Auto
}
}
// ---------------------------------------------------------------------------
// Status / probe types exposed to the frontend
// ---------------------------------------------------------------------------
#[derive(Serialize)]
pub struct TaggerModelStatus {
pub model_id: &'static str,
pub model_name: &'static str,
pub local_dir: String,
pub ready: bool,
pub missing_files: Vec<String>,
}
#[derive(Clone, Serialize)]
pub struct TaggerModelProgress {
pub total_files: usize,
pub completed_files: usize,
pub current_file: Option<String>,
pub done: bool,
}
// ---------------------------------------------------------------------------
// Runtime probe types exposed to the frontend
// ---------------------------------------------------------------------------
#[derive(Serialize)]
pub struct TaggerRuntimeProbe {
pub ready: bool,
pub acceleration: TaggerAcceleration,
pub session: TaggerRuntimeSessionProbe,
}
#[derive(Serialize)]
pub struct TaggerRuntimeSessionProbe {
pub file: &'static str,
pub inputs: Vec<String>,
pub outputs: Vec<String>,
}
// ---------------------------------------------------------------------------
// Tag record returned to callers
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, Serialize)]
pub struct TagResult {
pub tag: String,
pub confidence: f32,
}
#[derive(Debug, Clone, Serialize)]
pub struct TaggerOutput {
pub tags: Vec<TagResult>,
/// Highest-scoring rating label: "general" | "sensitive" | "questionable" | "explicit"
pub rating: String,
}
// ---------------------------------------------------------------------------
// Internal label table built from selected_tags.csv
// ---------------------------------------------------------------------------
#[derive(Debug, Clone)]
struct TagEntry {
name: String,
category: u8,
}
// ---------------------------------------------------------------------------
// Path helpers
// ---------------------------------------------------------------------------
pub fn model_dir(app_data_dir: &Path) -> PathBuf {
app_data_dir.join("models").join("wd-swinv2-tagger-v3")
}
// ---------------------------------------------------------------------------
// Settings persistence
// ---------------------------------------------------------------------------
pub fn tagger_acceleration(app_data_dir: &Path) -> TaggerAcceleration {
let path = app_data_dir.join(TAGGER_ACCELERATION_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return TaggerAcceleration::default();
};
match value.trim().to_ascii_lowercase().as_str() {
"cpu" => TaggerAcceleration::Cpu,
"directml" => TaggerAcceleration::Directml,
_ => TaggerAcceleration::Auto,
}
}
pub fn set_tagger_acceleration(
app_data_dir: &Path,
acceleration: TaggerAcceleration,
) -> Result<TaggerAcceleration> {
let path = app_data_dir.join(TAGGER_ACCELERATION_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, acceleration.as_str())?;
TAGGER_SESSION_DIRTY.store(true, Ordering::Relaxed);
Ok(acceleration)
}
pub fn tagger_threshold(app_data_dir: &Path) -> f32 {
let path = app_data_dir.join(TAGGER_THRESHOLD_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return DEFAULT_THRESHOLD;
};
value
.trim()
.parse::<f32>()
.unwrap_or(DEFAULT_THRESHOLD)
.clamp(0.01, 1.0)
}
pub fn set_tagger_threshold(app_data_dir: &Path, threshold: f32) -> Result<f32> {
let clamped = threshold.clamp(0.01, 1.0);
let path = app_data_dir.join(TAGGER_THRESHOLD_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, clamped.to_string())?;
Ok(clamped)
}
pub fn tagger_batch_size(app_data_dir: &Path) -> usize {
let path = app_data_dir.join(TAGGER_BATCH_SIZE_FILE);
let Ok(value) = std::fs::read_to_string(path) else {
return 8;
};
value
.trim()
.parse::<usize>()
.unwrap_or(8)
.clamp(1, 100)
}
pub fn set_tagger_batch_size(app_data_dir: &Path, batch_size: usize) -> Result<usize> {
let clamped = batch_size.clamp(1, 100);
let path = app_data_dir.join(TAGGER_BATCH_SIZE_FILE);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, clamped.to_string())?;
Ok(clamped)
}
// ---------------------------------------------------------------------------
// Model status / download
// ---------------------------------------------------------------------------
pub fn tagger_model_status(app_data_dir: &Path) -> TaggerModelStatus {
let local_dir = model_dir(app_data_dir);
// The ONNX runtime DLLs live in the caption model dir; reuse them.
let caption_model_dir = app_data_dir.join("models").join("florence-2-base-ft");
let missing_files = TAGGER_REQUIRED_FILES
.iter()
.filter(|file| {
let path = if file.starts_with("onnxruntime/") {
caption_model_dir.join(file)
} else {
local_dir.join(file)
};
!path.exists()
})
.map(|file| (*file).to_string())
.collect::<Vec<_>>();
TaggerModelStatus {
model_id: WD_TAGGER_MODEL_ID,
model_name: WD_TAGGER_MODEL_NAME,
local_dir: local_dir.to_string_lossy().to_string(),
ready: missing_files.is_empty(),
missing_files,
}
}
pub fn prepare_tagger_model_with_progress(
app_data_dir: &Path,
emit_progress: impl Fn(TaggerModelProgress),
) -> Result<TaggerModelStatus> {
let local_dir = model_dir(app_data_dir);
std::fs::create_dir_all(&local_dir)?;
// Only download the two tagger-specific files; ONNX runtime DLLs are
// already handled by the captioner download flow.
const DOWNLOAD_FILES: &[&str] = &["model.onnx", "selected_tags.csv"];
let api = Api::new()?;
let repo = api.repo(Repo::new(WD_TAGGER_MODEL_ID.to_string(), RepoType::Model));
let mut completed_files = DOWNLOAD_FILES
.iter()
.filter(|file| local_dir.join(file).exists())
.count();
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: None,
done: completed_files == DOWNLOAD_FILES.len(),
});
for file in DOWNLOAD_FILES {
let destination = local_dir.join(file);
if destination.exists() {
continue;
}
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: Some((*file).to_string()),
done: false,
});
let cached = repo.get(file)?;
std::fs::copy(cached, destination)?;
completed_files += 1;
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: Some((*file).to_string()),
done: completed_files == DOWNLOAD_FILES.len(),
});
}
emit_progress(TaggerModelProgress {
total_files: DOWNLOAD_FILES.len(),
completed_files,
current_file: None,
done: true,
});
Ok(tagger_model_status(app_data_dir))
}
pub fn delete_tagger_model(app_data_dir: &Path) -> Result<TaggerModelStatus> {
let local_dir = model_dir(app_data_dir);
if local_dir.exists() {
std::fs::remove_dir_all(&local_dir)?;
}
Ok(tagger_model_status(app_data_dir))
}
pub fn probe_tagger_runtime(app_data_dir: &Path) -> Result<TaggerRuntimeProbe> {
let status = tagger_model_status(app_data_dir);
if !status.ready {
anyhow::bail!(
"WD Tagger model is missing {} required file(s): {}",
status.missing_files.len(),
status.missing_files.join(", ")
);
}
let local_dir = model_dir(app_data_dir);
let caption_model_dir = app_data_dir.join("models").join("florence-2-base-ft");
crate::captioner::ensure_onnx_runtime(&caption_model_dir)?;
let acceleration = tagger_acceleration(app_data_dir);
let model_path = local_dir.join("model.onnx");
// Verify that the model file exists and has non-zero size before trying
// to create an ORT session (better error message on corruption).
let metadata = std::fs::metadata(&model_path)?;
if metadata.len() == 0 {
anyhow::bail!("model.onnx is empty");
}
// Actually create a session to verify the EP loads correctly.
let loaded_acceleration = match acceleration {
TaggerAcceleration::Cpu => {
create_tagger_session(&model_path, TaggerAcceleration::Cpu)?;
TaggerAcceleration::Cpu
}
TaggerAcceleration::Auto => {
// Try DirectML explicitly; if it fails the real session would have
// fallen back to CPU silently — report what would actually run.
let directml_ok =
create_tagger_session(&model_path, TaggerAcceleration::Directml).is_ok();
if directml_ok {
TaggerAcceleration::Directml
} else {
create_tagger_session(&model_path, TaggerAcceleration::Cpu)?;
TaggerAcceleration::Cpu
}
}
TaggerAcceleration::Directml => {
create_tagger_session(&model_path, TaggerAcceleration::Directml)?;
TaggerAcceleration::Directml
}
};
Ok(TaggerRuntimeProbe {
ready: true,
acceleration: loaded_acceleration,
session: TaggerRuntimeSessionProbe {
file: "model.onnx",
inputs: vec!["pixel_values".to_string()],
outputs: vec![format!("output [EP: {:?}]", loaded_acceleration)],
},
})
}
// ---------------------------------------------------------------------------
// Top-level inference entry point
// ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// Tagger implementation
// ---------------------------------------------------------------------------
pub struct WdTagger {
session: Session,
labels: Vec<TagEntry>,
threshold: f32,
input_size: usize,
}
impl WdTagger {
pub fn new(app_data_dir: &Path) -> Result<Self> {
let started_at = Instant::now();
let status = tagger_model_status(app_data_dir);
if !status.ready {
anyhow::bail!(
"WD tagger model is missing {} required file(s): {}",
status.missing_files.len(),
status.missing_files.join(", ")
);
}
let local_dir = model_dir(app_data_dir);
// The ONNX runtime DLLs are shared with the captioner; use the
// captioner's shared ORT init lock to avoid double-initialisation.
let caption_model_dir = app_data_dir.join("models").join("florence-2-base-ft");
crate::captioner::ensure_onnx_runtime(&caption_model_dir).map_err(|e| {
anyhow::anyhow!(
"ONNX Runtime not initialised — download the Florence-2 caption model first \
to get the shared runtime DLLs. Original error: {e}"
)
})?;
let acceleration = tagger_acceleration(app_data_dir);
let threshold = tagger_threshold(app_data_dir);
let model_path = local_dir.join("model.onnx");
let labels_path = local_dir.join("selected_tags.csv");
let session = create_tagger_session(&model_path, acceleration)?;
// Determine the input spatial size from the ONNX model graph.
// WD v3 models use (1, H, W, 3) where H == W, typically 448.
let input_size = {
let inputs = session.inputs();
let dim = inputs
.first()
.and_then(|inp| {
if let ort::value::ValueType::Tensor { shape, .. } = inp.dtype() {
shape
.get(1)
.and_then(|&d| if d > 0 { Some(d as usize) } else { None })
} else {
None
}
})
.unwrap_or(448);
dim
};
let labels = load_labels(&labels_path)?;
println!(
"WD tagger loaded in {:?} ({} labels, input {}x{}, {:?} acceleration)",
started_at.elapsed(),
labels.len(),
input_size,
input_size,
acceleration,
);
Ok(Self {
session,
labels,
threshold,
input_size,
})
}
pub fn run(&mut self, image_path: &Path, max_tags: usize) -> Result<TaggerOutput> {
let started_at = Instant::now();
let image_array = preprocess_image(image_path, self.input_size)?;
let batch_size = 1usize;
let input = Tensor::from_array((
[batch_size, self.input_size, self.input_size, 3usize],
image_array.into_boxed_slice(),
))
.map_err(|error| anyhow::anyhow!("{error}"))?;
let input_name: String = self.session.inputs()[0].name().to_string();
let outputs = self
.session
.run(ort::inputs! { input_name.as_str() => input })
.map_err(|error| anyhow::anyhow!("{error}"))?;
let (_, probabilities) = outputs[0]
.try_extract_tensor::<f32>()
.map_err(|error| anyhow::anyhow!("{error}"))?;
let probs: &[f32] = &probabilities;
if probs.len() != self.labels.len() {
anyhow::bail!(
"Model output length {} does not match label count {}",
probs.len(),
self.labels.len()
);
}
// Collect rating scores (category 9) - pick the argmax as the rating.
let rating = self
.labels
.iter()
.zip(probs.iter())
.filter(|(entry, _)| entry.category == RATING_CATEGORY)
.max_by(|(_, a), (_, b)| a.total_cmp(b))
.map(|(entry, _)| entry.name.clone())
.unwrap_or_else(|| "general".to_string());
// Collect general + character tags above threshold, sorted by confidence.
let mut tags: Vec<TagResult> = self
.labels
.iter()
.zip(probs.iter())
.filter(|(entry, prob)| {
(entry.category == GENERAL_CATEGORY || entry.category == CHARACTER_CATEGORY)
&& **prob >= self.threshold
})
.map(|(entry, prob)| TagResult {
tag: entry.name.clone(),
confidence: *prob,
})
.collect();
tags.sort_by(|a, b| b.confidence.total_cmp(&a.confidence));
tags.truncate(max_tags);
println!(
"WD tagger: {} tags (threshold={}, rating={}) in {:?} for {}",
tags.len(),
self.threshold,
rating,
started_at.elapsed(),
image_path.display(),
);
Ok(TaggerOutput { tags, rating })
}
}
// ---------------------------------------------------------------------------
// Session creation mirrors captioner's `create_session`
// ---------------------------------------------------------------------------
fn create_tagger_session(path: &Path, acceleration: TaggerAcceleration) -> Result<Session> {
let builder = Session::builder().map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_optimization_level(GraphOptimizationLevel::Level3)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let use_directml = matches!(
acceleration,
TaggerAcceleration::Auto | TaggerAcceleration::Directml
);
let builder = builder
.with_memory_pattern(!use_directml)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_parallel_execution(false)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let builder = builder
.with_intra_threads(1)
.map_err(|error| anyhow::anyhow!("{error}"))?;
let mut builder = match acceleration {
TaggerAcceleration::Cpu => {
println!("WD tagger: using CPU execution provider");
builder
}
TaggerAcceleration::Auto => builder
.with_execution_providers([ep::DirectML::default().build().fail_silently()])
.unwrap_or_else(|error| {
println!("WD tagger: DirectML unavailable, falling back to CPU");
error.recover()
}),
TaggerAcceleration::Directml => builder
.with_execution_providers([ep::DirectML::default().build().error_on_failure()])
.map_err(|error| anyhow::anyhow!("{error}"))?,
};
let session = builder
.commit_from_file(path)
.map_err(|error| anyhow::anyhow!("{error}"))?;
Ok(session)
}
// ---------------------------------------------------------------------------
// Label loading
// ---------------------------------------------------------------------------
fn load_labels(path: &Path) -> Result<Vec<TagEntry>> {
let mut reader = csv::Reader::from_path(path)?;
let mut entries = Vec::new();
for result in reader.records() {
let record = result?;
// CSV columns: tag_id, name, category, count
let name = record
.get(1)
.ok_or_else(|| anyhow::anyhow!("Missing name column in selected_tags.csv"))?
.replace('_', " ");
let category: u8 = record
.get(2)
.ok_or_else(|| anyhow::anyhow!("Missing category column in selected_tags.csv"))?
.parse()
.unwrap_or(0);
entries.push(TagEntry { name, category });
}
if entries.is_empty() {
anyhow::bail!("selected_tags.csv is empty or could not be parsed");
}
Ok(entries)
}
// ---------------------------------------------------------------------------
// Image preprocessing
// WD tagger expects: (1, H, W, 3) float32, raw [0,255] values, BGR channel
// order, padded to square with white (255,255,255).
// ---------------------------------------------------------------------------
fn preprocess_image(image_path: &Path, target_size: usize) -> Result<Vec<f32>> {
let image = ImageReader::open(image_path)?.decode()?;
// Composite any alpha channel onto a white background.
let image_rgba = image.to_rgba8();
let (width, height) = image_rgba.dimensions();
let mut canvas_rgba =
image::RgbaImage::from_pixel(width, height, image::Rgba([255, 255, 255, 255]));
image::imageops::overlay(&mut canvas_rgba, &image_rgba, 0, 0);
let image_rgb = DynamicImage::ImageRgba8(canvas_rgba).to_rgb8();
// Pad to square.
let max_dim = width.max(height);
let pad_left = (max_dim - width) / 2;
let pad_top = (max_dim - height) / 2;
let mut square = image::RgbImage::from_pixel(max_dim, max_dim, image::Rgb([255, 255, 255]));
image::imageops::overlay(&mut square, &image_rgb, pad_left as i64, pad_top as i64);
// Resize to model input size.
let resized = image::imageops::resize(
&square,
target_size as u32,
target_size as u32,
FilterType::CatmullRom,
);
// Flatten to (H, W, 3) float32 in BGR order, values in [0, 255].
let mut pixel_values = vec![0.0f32; target_size * target_size * 3];
for (x, y, pixel) in resized.enumerate_pixels() {
let base = (y as usize * target_size + x as usize) * 3;
// BGR order
pixel_values[base] = f32::from(pixel[2]); // B
pixel_values[base + 1] = f32::from(pixel[1]); // G
pixel_values[base + 2] = f32::from(pixel[0]); // R
}
Ok(pixel_values)
}
+160 -5
View File
@@ -80,7 +80,12 @@ pub fn upsert_caption_embedding(conn: &Connection, image_id: i64, embedding: &[f
Ok(()) Ok(())
} }
pub fn find_similar_image_ids(conn: &Connection, image_id: i64, limit: usize) -> Result<Vec<i64>> { pub fn find_similar_image_ids(
conn: &Connection,
image_id: i64,
limit: usize,
folder_id: Option<i64>,
) -> Result<Vec<i64>> {
let embedding: Vec<u8> = match conn.query_row( let embedding: Vec<u8> = match conn.query_row(
"SELECT embedding FROM image_vec WHERE image_id = ?1", "SELECT embedding FROM image_vec WHERE image_id = ?1",
[image_id], [image_id],
@@ -91,19 +96,39 @@ pub fn find_similar_image_ids(conn: &Connection, image_id: i64, limit: usize) ->
Err(error) => return Err(error.into()), Err(error) => return Err(error.into()),
}; };
if let Some(folder_id) = folder_id {
// Brute-force cosine scan scoped to the folder — avoids the KNN k=4096 limit
// and returns exact nearest neighbours within the folder.
let mut stmt = conn.prepare(
"SELECT v.image_id
FROM image_vec v
JOIN images i ON i.id = v.image_id
WHERE i.folder_id = ?2
AND v.image_id != ?3
ORDER BY vec_distance_cosine(v.embedding, vec_f32(?1)) ASC
LIMIT ?4",
)?;
let rows = stmt.query_map((&embedding, folder_id, image_id, limit as i64), |row| {
row.get::<_, i64>(0)
})?;
return Ok(rows.collect::<rusqlite::Result<Vec<_>>>()?);
}
// Global KNN search (no folder filter) — use the ANN index.
let mut stmt = conn.prepare( let mut stmt = conn.prepare(
"SELECT image_id "SELECT image_id
FROM image_vec FROM image_vec
WHERE embedding MATCH vec_f32(?1) WHERE embedding MATCH vec_f32(?1)
AND k = ?2", AND k = ?2",
)?; )?;
let rows = stmt.query_map((&embedding, (limit as i64) + 1), |row| row.get::<_, i64>(0))?; let rows = stmt
.query_map((&embedding, (limit + 1) as i64), |row| row.get::<_, i64>(0))?
.collect::<rusqlite::Result<Vec<_>>>()?;
let mut ids = Vec::new(); let mut ids = Vec::new();
for row in rows { for row in rows {
let candidate_id = row?; if row != image_id {
if candidate_id != image_id { ids.push(row);
ids.push(candidate_id);
} }
if ids.len() >= limit { if ids.len() >= limit {
break; break;
@@ -112,6 +137,102 @@ pub fn find_similar_image_ids(conn: &Connection, image_id: i64, limit: usize) ->
Ok(ids) Ok(ids)
} }
// pub fn find_similar_image_matches(
// conn: &Connection,
// image_id: i64,
// folder_id: Option<i64>,
// threshold: f32,
// offset: usize,
// limit: usize,
// ) -> Result<Vec<(i64, f32)>> {
// let embedding: Vec<u8> = match conn.query_row(
// "SELECT embedding FROM image_vec WHERE image_id = ?1",
// [image_id],
// |row| row.get(0),
// ) {
// Ok(embedding) => embedding,
// Err(SqliteError::QueryReturnedNoRows) => return Ok(Vec::new()),
// Err(error) => return Err(error.into()),
// };
// let query = match folder_id {
// Some(_) => {
// "SELECT v.image_id, vec_distance_cosine(v.embedding, vec_f32(?1)) AS distance
// FROM image_vec v
// JOIN images i ON i.id = v.image_id
// WHERE i.folder_id = ?2
// AND v.image_id != ?3
// AND vec_distance_cosine(v.embedding, vec_f32(?1)) <= ?4
// ORDER BY distance ASC
// LIMIT ?5 OFFSET ?6"
// }
// None => {
// "SELECT v.image_id, vec_distance_cosine(v.embedding, vec_f32(?1)) AS distance
// FROM image_vec v
// WHERE v.image_id != ?2
// AND vec_distance_cosine(v.embedding, vec_f32(?1)) <= ?3
// ORDER BY distance ASC
// LIMIT ?4 OFFSET ?5"
// }
// };
// let mut stmt = conn.prepare(query)?;
// match folder_id {
// Some(folder_id) => Ok(stmt
// .query_map(
// (
// &embedding,
// folder_id,
// image_id,
// threshold,
// limit as i64,
// offset as i64,
// ),
// |row| Ok((row.get::<_, i64>(0)?, row.get::<_, f32>(1)?)),
// )?
// .collect::<rusqlite::Result<Vec<_>>>()?),
// None => Ok(stmt
// .query_map(
// (&embedding, image_id, threshold, limit as i64, offset as i64),
// |row| Ok((row.get::<_, i64>(0)?, row.get::<_, f32>(1)?)),
// )?
// .collect::<rusqlite::Result<Vec<_>>>()?),
// }
// }
pub fn get_image_embedding(conn: &Connection, image_id: i64) -> Result<Option<Vec<f32>>> {
let embedding: Result<Vec<u8>, rusqlite::Error> = conn.query_row(
"SELECT embedding FROM image_vec WHERE image_id = ?1",
[image_id],
|row| row.get(0),
);
match embedding {
Ok(bytes) => Ok(Some(unpack_f32(&bytes))),
Err(SqliteError::QueryReturnedNoRows) => Ok(None),
Err(error) => Err(error.into()),
}
}
pub fn get_embedding_revision(conn: &Connection) -> Result<String> {
let count: i64 = conn.query_row("SELECT COUNT(*) FROM image_vec", [], |row| row.get(0))?;
let max_updated_at: Option<String> = conn.query_row(
"SELECT MAX(embedding_updated_at) FROM images WHERE embedding_status = 'ready'",
[],
|row| row.get(0),
)?;
Ok(format!("{}:{}", count, max_updated_at.unwrap_or_default()))
}
// fn image_ids_for_folder(
// conn: &Connection,
// folder_id: i64,
// ) -> Result<std::collections::HashSet<i64>> {
// let mut stmt = conn.prepare("SELECT id FROM images WHERE folder_id = ?1")?;
// let rows = stmt.query_map([folder_id], |row| row.get::<_, i64>(0))?;
// Ok(rows.collect::<rusqlite::Result<std::collections::HashSet<_>>>()?)
// }
/// Returns all stored image embeddings with their image IDs, optionally filtered to one folder. /// Returns all stored image embeddings with their image IDs, optionally filtered to one folder.
/// Each entry is `(image_id, normalized_f32_embedding)`. /// Each entry is `(image_id, normalized_f32_embedding)`.
pub fn get_all_image_embeddings_with_ids( pub fn get_all_image_embeddings_with_ids(
@@ -189,6 +310,40 @@ pub fn search_image_ids_by_embedding(
Ok(ids) Ok(ids)
} }
/// Brute-force cosine search scoped to a single folder, ordered by ascending distance.
/// Used for region-based similarity search where we want folder-scoped results.
pub fn search_image_ids_by_embedding_in_folder(
conn: &Connection,
embedding: &[f32],
folder_id: i64,
exclude_image_id: Option<i64>,
limit: usize,
) -> Result<Vec<i64>> {
if embedding.len() != CLIP_VECTOR_DIM {
return Err(anyhow!(
"expected {}-dimensional embedding, got {}",
CLIP_VECTOR_DIM,
embedding.len()
));
}
let packed = pack_f32(embedding);
let exclude_id = exclude_image_id.unwrap_or(-1);
let mut stmt = conn.prepare(
"SELECT v.image_id
FROM image_vec v
JOIN images i ON i.id = v.image_id
WHERE i.folder_id = ?2
AND v.image_id != ?3
ORDER BY vec_distance_cosine(v.embedding, vec_f32(?1)) ASC
LIMIT ?4",
)?;
let rows = stmt.query_map((&packed, folder_id, exclude_id, limit as i64), |row| {
row.get::<_, i64>(0)
})?;
Ok(rows.collect::<rusqlite::Result<Vec<_>>>()?)
}
#[allow(dead_code)] #[allow(dead_code)]
pub fn search_caption_ids_by_embedding( pub fn search_caption_ids_by_embedding(
conn: &Connection, conn: &Connection,
+6 -4
View File
@@ -4,9 +4,9 @@
"version": "0.1.0", "version": "0.1.0",
"identifier": "wtf.jezz.phokus", "identifier": "wtf.jezz.phokus",
"build": { "build": {
"beforeDevCommand": "pnpm dev", "beforeDevCommand": "pnpm dev:vite",
"devUrl": "http://localhost:1420", "devUrl": "http://localhost:1420",
"beforeBuildCommand": "pnpm build", "beforeBuildCommand": "pnpm build:vite",
"frontendDist": "../dist" "frontendDist": "../dist"
}, },
"app": { "app": {
@@ -25,7 +25,9 @@
"csp": null, "csp": null,
"assetProtocol": { "assetProtocol": {
"enable": true, "enable": true,
"scope": ["**"] "scope": [
"**"
]
} }
} }
}, },
@@ -40,4 +42,4 @@
"icons/icon.ico" "icons/icon.ico"
] ]
} }
} }
+8
View File
@@ -6,6 +6,7 @@ import { Toolbar } from "./components/Toolbar";
import { Gallery } from "./components/Gallery"; import { Gallery } from "./components/Gallery";
import { Lightbox } from "./components/Lightbox"; import { Lightbox } from "./components/Lightbox";
import { TagCloud } from "./components/TagCloud"; import { TagCloud } from "./components/TagCloud";
import { DuplicateFinder } from "./components/DuplicateFinder";
import { TitleBar } from "./components/TitleBar"; import { TitleBar } from "./components/TitleBar";
import { SettingsModal } from "./components/SettingsModal"; import { SettingsModal } from "./components/SettingsModal";
@@ -14,6 +15,7 @@ export default function App() {
const loadBackgroundJobProgress = useGalleryStore((state) => state.loadBackgroundJobProgress); const loadBackgroundJobProgress = useGalleryStore((state) => state.loadBackgroundJobProgress);
const loadImages = useGalleryStore((state) => state.loadImages); const loadImages = useGalleryStore((state) => state.loadImages);
const loadCaptionModelStatus = useGalleryStore((state) => state.loadCaptionModelStatus); const loadCaptionModelStatus = useGalleryStore((state) => state.loadCaptionModelStatus);
const loadDuplicateScanCache = useGalleryStore((state) => state.loadDuplicateScanCache);
const subscribeToProgress = useGalleryStore((state) => state.subscribeToProgress); const subscribeToProgress = useGalleryStore((state) => state.subscribeToProgress);
const activeView = useGalleryStore((state) => state.activeView); const activeView = useGalleryStore((state) => state.activeView);
@@ -21,6 +23,7 @@ export default function App() {
loadFolders().then(() => { loadFolders().then(() => {
void loadBackgroundJobProgress(); void loadBackgroundJobProgress();
void loadCaptionModelStatus(); void loadCaptionModelStatus();
void loadDuplicateScanCache();
return loadImages(true); return loadImages(true);
}); });
let unlisten: (() => void) | undefined; let unlisten: (() => void) | undefined;
@@ -46,6 +49,11 @@ export default function App() {
<BackgroundTasks /> <BackgroundTasks />
<TagCloud /> <TagCloud />
</> </>
) : activeView === "duplicates" ? (
<>
<BackgroundTasks />
<DuplicateFinder />
</>
) : ( ) : (
<> <>
<Toolbar /> <Toolbar />
+57 -25
View File
@@ -58,6 +58,8 @@ export function BackgroundTasks() {
const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress); const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress);
const retryFailedEmbeddings = useGalleryStore((state) => state.retryFailedEmbeddings); const retryFailedEmbeddings = useGalleryStore((state) => state.retryFailedEmbeddings);
const clearTaggingJobs = useGalleryStore((state) => state.clearTaggingJobs); const clearTaggingJobs = useGalleryStore((state) => state.clearTaggingJobs);
const duplicateScanning = useGalleryStore((state) => state.duplicateScanning);
const duplicateScanProgress = useGalleryStore((state) => state.duplicateScanProgress);
const [expanded, setExpanded] = useState(false); const [expanded, setExpanded] = useState(false);
const [dismissed, setDismissed] = useState<Record<number, string>>({}); const [dismissed, setDismissed] = useState<Record<number, string>>({});
const [paused, setPaused] = useState<Record<number, Record<WorkerKey, boolean>>>({}); const [paused, setPaused] = useState<Record<number, Record<WorkerKey, boolean>>>({});
@@ -127,6 +129,7 @@ export function BackgroundTasks() {
}; };
const dismissTask = (id: number, snapshot: string) => { const dismissTask = (id: number, snapshot: string) => {
if (id < 0) return; // system tasks (duplicate scan) cannot be dismissed
void clearTaggingJobs(id); void clearTaggingJobs(id);
setDismissed((prev) => ({ ...prev, [id]: snapshot })); setDismissed((prev) => ({ ...prev, [id]: snapshot }));
setExpanded(false); setExpanded(false);
@@ -244,10 +247,35 @@ export function BackgroundTasks() {
.filter((t) => dismissed[t.id] !== t.snapshot); .filter((t) => dismissed[t.id] !== t.snapshot);
}, [folders, indexingProgress, mediaJobProgress, dismissed]); }, [folders, indexingProgress, mediaJobProgress, dismissed]);
if (tasks.length === 0) return null; // Synthetic task for duplicate scanning — negative id so dismiss/retry are suppressed
const duplicateScanTask: Task | null = duplicateScanning ? {
id: -1,
name: "Duplicate Scan",
stages: [{
label: "Hashing",
detail: duplicateScanProgress
? `${duplicateScanProgress.scanned.toLocaleString()} / ${duplicateScanProgress.total.toLocaleString()}`
: "Starting…",
progress: duplicateScanProgress && duplicateScanProgress.total > 0
? (duplicateScanProgress.scanned / duplicateScanProgress.total) * 100
: null,
failed: false,
}],
hasFailedEmbeddings: false,
hasFailedTagging: false,
pendingMediaWork: 1,
embeddingProcessed: 0,
embeddingTotal: 0,
currentFile: null,
snapshot: "",
} : null;
const primary = tasks[0]; const allTasks = duplicateScanTask ? [duplicateScanTask, ...tasks] : tasks;
const extraCount = tasks.length - 1;
if (allTasks.length === 0) return null;
const primary = allTasks[0];
const extraCount = allTasks.length - 1;
const hasFailed = tasks.some((t) => (t.hasFailedEmbeddings || t.hasFailedTagging) && t.pendingMediaWork === 0); const hasFailed = tasks.some((t) => (t.hasFailedEmbeddings || t.hasFailedTagging) && t.pendingMediaWork === 0);
// Best progress bar value: use embedding progress if available (most informative), // Best progress bar value: use embedding progress if available (most informative),
@@ -348,8 +376,8 @@ export function BackgroundTasks() {
</button> </button>
)} )}
{/* Expand chevron (only when multiple folders) */} {/* Expand chevron (only when multiple tasks) */}
{tasks.length > 1 && ( {allTasks.length > 1 && (
<svg <svg
className={`h-3.5 w-3.5 text-gray-600 transition-transform duration-200 shrink-0 ${expanded ? "rotate-180" : ""}`} className={`h-3.5 w-3.5 text-gray-600 transition-transform duration-200 shrink-0 ${expanded ? "rotate-180" : ""}`}
fill="none" viewBox="0 0 24 24" stroke="currentColor" fill="none" viewBox="0 0 24 24" stroke="currentColor"
@@ -358,22 +386,24 @@ export function BackgroundTasks() {
</svg> </svg>
)} )}
{/* Dismiss */} {/* Dismiss — hidden for system tasks like duplicate scan */}
<button {primary.id >= 0 && (
className="p-1 rounded-md text-gray-600 hover:text-gray-300 hover:bg-white/8 transition-colors shrink-0" <button
title="Dismiss" className="p-1 rounded-md text-gray-600 hover:text-gray-300 hover:bg-white/8 transition-colors shrink-0"
onClick={(e) => { e.stopPropagation(); dismissTask(primary.id, primary.snapshot); }} title="Dismiss"
> onClick={(e) => { e.stopPropagation(); dismissTask(primary.id, primary.snapshot); }}
<svg className="h-3.5 w-3.5" fill="none" viewBox="0 0 24 24" stroke="currentColor"> >
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" /> <svg className="h-3.5 w-3.5" fill="none" viewBox="0 0 24 24" stroke="currentColor">
</svg> <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</button> </svg>
</button>
)}
</div> </div>
{/* Expanded panel — one row per folder */} {/* Expanded panel — one row per folder */}
{expanded && ( {expanded && (
<div className="border-t border-white/[0.06] bg-white/[0.02] px-5 py-3 space-y-3"> <div className="border-t border-white/[0.06] bg-white/[0.02] px-5 py-3 space-y-3">
{tasks.map((task) => { {allTasks.map((task) => {
const taskEmbeddingStage = task.stages.find((s) => s.label === "Embeddings"); const taskEmbeddingStage = task.stages.find((s) => s.label === "Embeddings");
const taskTaggingStage = task.stages.find((s) => s.label === "Tags"); const taskTaggingStage = task.stages.find((s) => s.label === "Tags");
const taskScanningStage = task.stages.find((s) => s.label === "Scanning"); const taskScanningStage = task.stages.find((s) => s.label === "Scanning");
@@ -453,15 +483,17 @@ export function BackgroundTasks() {
</button> </button>
)} )}
<button {task.id >= 0 && (
className="p-1 rounded-md text-gray-600 hover:text-gray-300 hover:bg-white/8 transition-colors shrink-0" <button
title="Dismiss" className="p-1 rounded-md text-gray-600 hover:text-gray-300 hover:bg-white/8 transition-colors shrink-0"
onClick={() => dismissTask(task.id, task.snapshot)} title="Dismiss"
> onClick={() => dismissTask(task.id, task.snapshot)}
<svg className="h-3 w-3" fill="none" viewBox="0 0 24 24" stroke="currentColor"> >
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" /> <svg className="h-3 w-3" fill="none" viewBox="0 0 24 24" stroke="currentColor">
</svg> <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</button> </svg>
</button>
)}
</div> </div>
{task.currentFile && ( {task.currentFile && (
+274
View File
@@ -0,0 +1,274 @@
import { useState } from "react";
import { convertFileSrc } from "@tauri-apps/api/core";
import { DuplicateGroup, useGalleryStore } from "../store";
function formatBytes(bytes: number): string {
if (bytes >= 1_073_741_824) return `${(bytes / 1_073_741_824).toFixed(1)} GB`;
if (bytes >= 1_048_576) return `${(bytes / 1_048_576).toFixed(1)} MB`;
if (bytes >= 1024) return `${(bytes / 1024).toFixed(0)} KB`;
return `${bytes} B`;
}
function DuplicateGroupCard({ group }: { group: DuplicateGroup }) {
const selectedIds = useGalleryStore((state) => state.duplicateSelectedIds);
const toggleDuplicateSelected = useGalleryStore((state) => state.toggleDuplicateSelected);
const selectAllDuplicates = useGalleryStore((state) => state.selectAllDuplicates);
const groupSelectedCount = group.images.filter((img) => selectedIds.has(img.id)).length;
const noneSelected = groupSelectedCount === 0;
// "Keep all but the first" — a common quick action
const handleKeepFirst = () => {
const toDelete = group.images.slice(1).map((img) => img.id);
// Clear any selection for this group first, then add the ones to delete
for (const img of group.images) {
if (selectedIds.has(img.id)) toggleDuplicateSelected(img.id);
}
selectAllDuplicates(toDelete);
};
return (
<div className="rounded-2xl border border-white/[0.07] bg-white/[0.02] p-4">
{/* Group header */}
<div className="mb-3 flex items-center justify-between gap-3">
<div className="flex items-center gap-2">
<span className="rounded-md border border-white/10 bg-white/[0.04] px-2 py-0.5 text-[11px] text-gray-400">
{group.images.length} copies
</span>
<span className="text-[11px] text-white/30">{formatBytes(group.file_size)} each</span>
<span className="text-[11px] text-white/20">
{formatBytes(group.file_size * (group.images.length - 1))} wasted
</span>
</div>
<div className="flex items-center gap-2">
{noneSelected ? (
<button
className="text-[11px] text-white/35 transition-colors hover:text-white/70"
onClick={handleKeepFirst}
>
Keep first
</button>
) : (
<button
className="text-[11px] text-white/35 transition-colors hover:text-white/70"
onClick={() => {
for (const img of group.images) {
if (selectedIds.has(img.id)) toggleDuplicateSelected(img.id);
}
}}
>
Deselect all
</button>
)}
</div>
</div>
{/* Image grid */}
<div className="flex flex-wrap gap-3">
{group.images.map((image) => {
const isSelected = selectedIds.has(image.id);
const src = image.thumbnail_path ? convertFileSrc(image.thumbnail_path) : null;
return (
<button
key={image.id}
className={`group relative overflow-hidden rounded-xl border transition-all ${
isSelected
? "border-red-400/50 ring-1 ring-red-400/30"
: "border-white/8 hover:border-white/20"
}`}
style={{ width: 140, height: 105 }}
onClick={() => toggleDuplicateSelected(image.id)}
title={image.path}
>
{src ? (
<img src={src} alt="" className="h-full w-full object-cover" draggable={false} />
) : (
<div className="h-full w-full bg-white/[0.03]" />
)}
{/* Delete overlay */}
{isSelected ? (
<div className="absolute inset-0 flex items-center justify-center bg-red-950/60">
<svg className="h-6 w-6 text-red-300" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M19 7l-.867 12.142A2 2 0 0116.138 21H7.862a2 2 0 01-1.995-1.858L5 7m5 4v6m4-6v6m1-10V4a1 1 0 00-1-1h-4a1 1 0 00-1 1v3M4 7h16" />
</svg>
</div>
) : null}
{/* Path tooltip on hover */}
<div className="absolute inset-x-0 bottom-0 bg-gradient-to-t from-black/90 to-transparent px-2 pb-1.5 pt-4 opacity-0 transition-opacity group-hover:opacity-100">
<p className="truncate text-[9px] text-white/60">{image.path.split(/[\\/]/).slice(-2).join("/")}</p>
</div>
</button>
);
})}
</div>
</div>
);
}
function formatRelativeTime(unixSecs: number): string {
const diff = Math.floor(Date.now() / 1000) - unixSecs;
if (diff < 60) return "just now";
if (diff < 3600) return `${Math.floor(diff / 60)}m ago`;
if (diff < 86400) return `${Math.floor(diff / 3600)}h ago`;
return `${Math.floor(diff / 86400)}d ago`;
}
export function DuplicateFinder() {
const duplicateGroups = useGalleryStore((state) => state.duplicateGroups);
const duplicateScanning = useGalleryStore((state) => state.duplicateScanning);
const duplicateScanProgress = useGalleryStore((state) => state.duplicateScanProgress);
const duplicateSelectedIds = useGalleryStore((state) => state.duplicateSelectedIds);
const duplicateLastScanned = useGalleryStore((state) => state.duplicateLastScanned);
const selectedFolderId = useGalleryStore((state) => state.selectedFolderId);
const scanDuplicates = useGalleryStore((state) => state.scanDuplicates);
const clearDuplicateSelection = useGalleryStore((state) => state.clearDuplicateSelection);
const selectKeepFirstAllGroups = useGalleryStore((state) => state.selectKeepFirstAllGroups);
const deleteSelectedDuplicates = useGalleryStore((state) => state.deleteSelectedDuplicates);
const [deleting, setDeleting] = useState(false);
const [deleteResult, setDeleteResult] = useState<string | null>(null);
const selectedCount = duplicateSelectedIds.size;
const hasResults = duplicateGroups.length > 0;
const hasScanned = hasResults || duplicateLastScanned !== null || (!duplicateScanning && duplicateScanProgress !== null);
const totalWasted = duplicateGroups.reduce(
(sum, g) => sum + g.file_size * (g.images.length - 1),
0,
);
const totalDuplicateImages = duplicateGroups.reduce((sum, g) => sum + g.images.length - 1, 0);
const handleDelete = async () => {
setDeleting(true);
setDeleteResult(null);
try {
const deleted = await deleteSelectedDuplicates();
setDeleteResult(`Deleted ${deleted} file${deleted === 1 ? "" : "s"}.`);
} catch (e) {
setDeleteResult(String(e));
} finally {
setDeleting(false);
}
};
const progressPercent =
duplicateScanProgress && duplicateScanProgress.total > 0
? Math.round((duplicateScanProgress.scanned / duplicateScanProgress.total) * 100)
: 0;
return (
<div className="flex min-h-0 flex-1 flex-col overflow-hidden bg-[#07080f]">
{/* Header */}
<div className="shrink-0 border-b border-white/[0.05] px-6 py-4">
<div className="flex items-center justify-between gap-4">
<div>
<h2 className="text-[15px] font-semibold text-white">Duplicate Finder</h2>
<p className="mt-0.5 text-[11px] text-white/30">
{duplicateScanning
? duplicateScanProgress
? `Scanning… ${duplicateScanProgress.scanned.toLocaleString()} / ${duplicateScanProgress.total.toLocaleString()}`
: "Starting scan…"
: hasResults
? `${duplicateGroups.length} group${duplicateGroups.length === 1 ? "" : "s"} · ${formatBytes(totalWasted)} reclaimable`
: duplicateLastScanned !== null
? "No duplicates found"
: "Scan your library for identical files"}
</p>
{!duplicateScanning && duplicateLastScanned !== null && (
<p className="mt-0.5 text-[10px] text-white/20">
Last scanned {formatRelativeTime(duplicateLastScanned)}
</p>
)}
</div>
<div className="flex items-center gap-2">
{/* Batch select — only shown when there are groups and nothing is selected yet */}
{hasResults && selectedCount === 0 && !deleting && (
<button
className="rounded-lg border border-white/10 bg-white/[0.04] px-3 py-1.5 text-xs text-gray-400 transition-colors hover:bg-white/[0.07] hover:text-white"
onClick={selectKeepFirstAllGroups}
title={`Mark ${totalDuplicateImages} duplicate${totalDuplicateImages === 1 ? "" : "s"} for deletion across all groups (keeps first in each)`}
>
Select all duplicates
</button>
)}
{selectedCount > 0 ? (
<>
<span className="text-[11px] text-white/40">{selectedCount} marked for deletion</span>
<button
className="rounded-lg border border-white/10 bg-white/[0.04] px-3 py-1.5 text-xs text-gray-400 transition-colors hover:bg-white/[0.07] hover:text-white disabled:opacity-40"
onClick={clearDuplicateSelection}
disabled={deleting}
>
Deselect all
</button>
<button
className="rounded-lg border border-red-400/25 bg-red-500/10 px-3 py-1.5 text-xs text-red-300 transition-colors hover:bg-red-500/15 disabled:cursor-not-allowed disabled:opacity-40"
onClick={handleDelete}
disabled={deleting}
>
{deleting ? "Deleting…" : `Delete ${selectedCount} file${selectedCount === 1 ? "" : "s"}`}
</button>
</>
) : null}
<button
className="rounded-lg border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-40"
onClick={() => { setDeleteResult(null); void scanDuplicates(selectedFolderId); }}
disabled={duplicateScanning}
>
{duplicateScanning ? "Scanning…" : hasScanned ? "Rescan" : "Scan for duplicates"}
</button>
</div>
</div>
{/* Progress bar */}
{duplicateScanning && duplicateScanProgress ? (
<div className="mt-3 h-px overflow-hidden rounded-full bg-white/[0.07]">
<div
className="h-full rounded-full bg-blue-500/60 transition-[width] duration-200"
style={{ width: `${progressPercent}%` }}
/>
</div>
) : null}
{deleteResult ? (
<p className="mt-2 text-[11px] text-white/40">{deleteResult}</p>
) : null}
</div>
{/* Body */}
{duplicateScanning && !hasResults ? (
<div className="flex flex-1 items-center justify-center gap-3 text-white/25">
<div className="h-5 w-5 animate-spin rounded-full border-2 border-white/15 border-t-white/50" />
<span className="text-sm">Hashing files</span>
</div>
) : !hasScanned ? (
<div className="flex flex-1 items-center justify-center px-8">
<div className="max-w-sm text-center">
<svg className="mx-auto mb-4 h-10 w-10 text-white/10" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1}
d="M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z" />
</svg>
<p className="text-sm text-white/30">
Finds files with identical content regardless of filename or location.
Click <strong className="text-white/50">Scan for duplicates</strong> to begin.
</p>
<p className="mt-2 text-xs text-white/20">
Large libraries may take a minute files are hashed from disk.
</p>
</div>
</div>
) : duplicateGroups.length === 0 ? (
<div className="flex flex-1 items-center justify-center">
<p className="text-sm text-white/25">No duplicate files found.</p>
</div>
) : (
<div className="overflow-y-auto px-6 py-5">
<div className="space-y-4">
{duplicateGroups.map((group) => (
<DuplicateGroupCard key={group.file_hash} group={group} />
))}
</div>
</div>
)}
</div>
);
}
+94 -80
View File
@@ -1,6 +1,6 @@
import { useEffect, useRef, useCallback, useState } from "react"; import { useEffect, useRef, useCallback, useState } from "react";
import { convertFileSrc } from "@tauri-apps/api/core"; import { convertFileSrc } from "@tauri-apps/api/core";
import { ImageRecord, tileSizeForZoom, useGalleryStore } from "../store"; import { ImageRecord, parseSearchValue, tileSizeForZoom, useGalleryStore } from "../store";
const GAP = 6; const GAP = 6;
@@ -30,6 +30,7 @@ function ContextMenu({
const openImage = useGalleryStore((state) => state.openImage); const openImage = useGalleryStore((state) => state.openImage);
const updateImageDetails = useGalleryStore((state) => state.updateImageDetails); const updateImageDetails = useGalleryStore((state) => state.updateImageDetails);
const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages); const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages);
const similarScope = useGalleryStore((state) => state.similarScope);
const canFindSimilar = image.embedding_status === "ready"; const canFindSimilar = image.embedding_status === "ready";
return ( return (
@@ -59,7 +60,7 @@ function ContextMenu({
}`} }`}
onClick={async () => { onClick={async () => {
if (!canFindSimilar) return; if (!canFindSimilar) return;
await loadSimilarImages(image.id); await loadSimilarImages(image.id, similarScope === "current_folder" ? image.folder_id : null, true, image.folder_id);
onClose(); onClose();
}} }}
disabled={!canFindSimilar} disabled={!canFindSimilar}
@@ -115,6 +116,7 @@ function ImageTile({
const [loaded, setLoaded] = useState(false); const [loaded, setLoaded] = useState(false);
const [errored, setErrored] = useState(false); const [errored, setErrored] = useState(false);
const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages); const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages);
const similarScope = useGalleryStore((state) => state.similarScope);
const canFindSimilar = image.embedding_status === "ready"; const canFindSimilar = image.embedding_status === "ready";
const src = image.thumbnail_path const src = image.thumbnail_path
@@ -181,6 +183,15 @@ function ImageTile({
</svg> </svg>
</div> </div>
)} )}
{image.rating > 0 && (
<div className="flex items-center gap-0.5 rounded-md bg-black/60 px-1.5 py-1 text-amber-300 backdrop-blur-sm">
{Array.from({ length: image.rating }, (_, index) => (
<svg key={index} className="h-2.5 w-2.5" fill="currentColor" viewBox="0 0 20 20">
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.176 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81H7.03a1 1 0 00.951-.69l1.07-3.292z" />
</svg>
))}
</div>
)}
{image.media_kind === "video" && image.duration_ms && ( {image.media_kind === "video" && image.duration_ms && (
<div className="rounded-md bg-black/60 px-1.5 py-0.5 text-[10px] font-medium text-white/80 backdrop-blur-sm"> <div className="rounded-md bg-black/60 px-1.5 py-0.5 text-[10px] font-medium text-white/80 backdrop-blur-sm">
{formatDuration(image.duration_ms)} {formatDuration(image.duration_ms)}
@@ -230,7 +241,7 @@ function ImageTile({
onClick={(event) => { onClick={(event) => {
event.stopPropagation(); event.stopPropagation();
if (!canFindSimilar) return; if (!canFindSimilar) return;
void loadSimilarImages(image.id); void loadSimilarImages(image.id, similarScope === "current_folder" ? image.folder_id : null, true, image.folder_id);
}} }}
disabled={!canFindSimilar} disabled={!canFindSimilar}
> >
@@ -250,11 +261,11 @@ export function Gallery() {
const loadingImages = useGalleryStore((state) => state.loadingImages); const loadingImages = useGalleryStore((state) => state.loadingImages);
const zoomPreset = useGalleryStore((state) => state.zoomPreset); const zoomPreset = useGalleryStore((state) => state.zoomPreset);
const search = useGalleryStore((state) => state.search); const search = useGalleryStore((state) => state.search);
const searchMode = useGalleryStore((state) => state.searchMode);
const collectionTitle = useGalleryStore((state) => state.collectionTitle); const collectionTitle = useGalleryStore((state) => state.collectionTitle);
const imageLoadError = useGalleryStore((state) => state.imageLoadError); const imageLoadError = useGalleryStore((state) => state.imageLoadError);
const galleryScrollResetKey = useGalleryStore((state) => state.galleryScrollResetKey); const galleryScrollResetKey = useGalleryStore((state) => state.galleryScrollResetKey);
const isSimilarResults = collectionTitle === "Similar Images"; const isSimilarResults = collectionTitle === "Similar Images";
const parsedSearch = parseSearchValue(search);
const parentRef = useRef<HTMLDivElement>(null); const parentRef = useRef<HTMLDivElement>(null);
const [contextMenu, setContextMenu] = useState<{ x: number; y: number; image: ImageRecord } | null>(null); const [contextMenu, setContextMenu] = useState<{ x: number; y: number; image: ImageRecord } | null>(null);
@@ -262,6 +273,7 @@ export function Gallery() {
const handleScroll = useCallback(() => { const handleScroll = useCallback(() => {
const element = parentRef.current; const element = parentRef.current;
if (!element) return; if (!element) return;
if (element.scrollTop < 24) return;
const nearBottom = element.scrollTop + element.clientHeight >= element.scrollHeight - 600; const nearBottom = element.scrollTop + element.clientHeight >= element.scrollHeight - 600;
if (nearBottom && !loadingImages && images.length < totalImages) { if (nearBottom && !loadingImages && images.length < totalImages) {
void loadMoreImages(); void loadMoreImages();
@@ -295,85 +307,87 @@ export function Gallery() {
}; };
}, []); }, []);
if (images.length === 0 && loadingImages) {
return (
<div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8">
<div className="rounded-2xl border border-white/[0.06] bg-white/[0.02] p-8 min-w-72">
<div className="h-5 w-5 mx-auto rounded-full border-2 border-white/20 border-t-white/60 animate-spin" />
<p className="mt-4 text-sm text-white/40 font-medium">
{isSimilarResults
? "Finding similar images"
: searchMode === "semantic" && search.trim().length > 0
? `Searching for matches to "${search}"`
: "Loading media"}
</p>
<p className="text-xs text-white/20 mt-1">
{isSimilarResults
? "Comparing visual embeddings"
: searchMode === "semantic" && search.trim().length > 0
? "Semantic search can take a little longer than filename search"
: "Fetching results"}
</p>
</div>
</div>
);
}
if (images.length === 0 && !loadingImages) {
return (
<div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8">
<div className="rounded-2xl border border-white/[0.06] bg-white/[0.02] p-8">
<svg className="h-12 w-12 mx-auto text-white/10 mb-4" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={0.75}
d="M4 16l4.586-4.586a2 2 0 012.828 0L16 16m-2-2l1.586-1.586a2 2 0 012.828 0L20 14m-6-6h.01M6 20h12a2 2 0 002-2V6a2 2 0 00-2-2H6a2 2 0 00-2 2v12a2 2 0 002 2z" />
</svg>
<p className="text-sm text-white/30 font-medium">
{imageLoadError
? "Could not load results"
: isSimilarResults
? "No similar images found"
: searchMode === "semantic" && search.trim().length > 0
? "No semantic matches found"
: "No media found"}
</p>
<p className="text-xs text-white/15 mt-1">
{imageLoadError
? imageLoadError
: isSimilarResults
? "This item may be visually isolated, or more embeddings may need to finish processing"
: searchMode === "semantic" && search.trim().length > 0
? "Try a broader phrase, or wait for more embeddings to finish processing"
: "Try adjusting your filters or add a new folder"}
</p>
</div>
</div>
);
}
return ( return (
<div ref={parentRef} className="relative flex-1 overflow-y-auto overflow-x-hidden min-h-0 bg-[#07080f]"> <div ref={parentRef} className="relative flex-1 overflow-y-auto overflow-x-hidden min-h-0 bg-[#07080f]">
<div {images.length === 0 && loadingImages ? (
className="grid content-start" <div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8 absolute inset-0">
style={{ <div className="rounded-2xl border border-white/[0.06] bg-white/[0.02] p-8 min-w-72">
padding: GAP, <div className="h-5 w-5 mx-auto rounded-full border-2 border-white/20 border-t-white/60 animate-spin" />
gap: GAP, <p className="mt-4 text-sm text-white/40 font-medium">
gridTemplateColumns: `repeat(auto-fill, minmax(${tileSizeForZoom(zoomPreset)}px, 1fr))`, {isSimilarResults
}} ? "Finding similar images"
> : parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
{images.map((image) => ( ? `Searching for matches to "${parsedSearch.query}"`
<ImageTile : parsedSearch.mode === "tag" && parsedSearch.query.length > 0
key={image.id} ? `Searching tags for "${parsedSearch.query}"`
image={image} : "Loading media"}
onClick={() => openImage(image)} </p>
onContextMenu={(event) => { <p className="text-xs text-white/20 mt-1">
event.preventDefault(); {isSimilarResults
setContextMenu({ x: event.clientX, y: event.clientY, image }); ? "Comparing visual embeddings"
}} : parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
/> ? "Semantic search can take a little longer than filename search"
))} : parsedSearch.mode === "tag" && parsedSearch.query.length > 0
</div> ? "Matching against AI and user tags"
: "Fetching results"}
</p>
</div>
</div>
) : images.length === 0 && !loadingImages ? (
<div className="flex flex-1 flex-col items-center justify-center gap-4 text-center px-8 absolute inset-0">
<div className="rounded-2xl border border-white/[0.06] bg-white/[0.02] p-8">
<svg className="h-12 w-12 mx-auto text-white/10 mb-4" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={0.75}
d="M4 16l4.586-4.586a2 2 0 012.828 0L16 16m-2-2l1.586-1.586a2 2 0 012.828 0L20 14m-6-6h.01M6 20h12a2 2 0 002-2V6a2 2 0 00-2-2H6a2 2 0 00-2 2v12a2 2 0 002 2z" />
</svg>
<p className="text-sm text-white/30 font-medium">
{imageLoadError
? "Could not load results"
: isSimilarResults
? "No similar images found"
: parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
? "No semantic matches found"
: parsedSearch.mode === "tag" && parsedSearch.query.length > 0
? "No tag matches found"
: "No media found"}
</p>
<p className="text-xs text-white/15 mt-1">
{imageLoadError
? imageLoadError
: isSimilarResults
? "This item may be visually isolated, or more embeddings may need to finish processing"
: parsedSearch.mode === "semantic" && parsedSearch.query.length > 0
? "Try a broader phrase, or wait for more embeddings to finish processing"
: parsedSearch.mode === "tag" && parsedSearch.query.length > 0
? "Try a shorter tag, or wait for more tagging jobs to finish"
: "Try adjusting your filters or add a new folder"}
</p>
</div>
</div>
) : (
<div
className="grid content-start"
style={{
padding: GAP,
gap: GAP,
gridTemplateColumns: `repeat(auto-fill, minmax(${tileSizeForZoom(zoomPreset)}px, 1fr))`,
}}
>
{images.map((image) => (
<ImageTile
key={image.id}
image={image}
onClick={() => openImage(image)}
onContextMenu={(event) => {
event.preventDefault();
setContextMenu({ x: event.clientX, y: event.clientY, image });
}}
/>
))}
</div>
)}
{loadingImages ? ( {images.length > 0 && loadingImages ? (
<div className="flex justify-center py-8"> <div className="flex justify-center py-8">
<div className="h-4 w-4 rounded-full border-2 border-white/20 border-t-white/60 animate-spin" /> <div className="h-4 w-4 rounded-full border-2 border-white/20 border-t-white/60 animate-spin" />
</div> </div>
+259 -25
View File
@@ -58,28 +58,96 @@ function ratingPill(rating: AiRating): { label: string; className: string } {
} }
} }
interface DragRect {
startX: number;
startY: number;
endX: number;
endY: number;
}
/** Compute a CSS-pixel rect (relative to the viewport container) from a DragRect. */
function normaliseRect(r: DragRect): { left: number; top: number; width: number; height: number } {
return {
left: Math.min(r.startX, r.endX),
top: Math.min(r.startY, r.endY),
width: Math.abs(r.endX - r.startX),
height: Math.abs(r.endY - r.startY),
};
}
/** Convert a CSS-pixel drag rect (relative to viewport container) to normalised 01 crop coords
* relative to the actual rendered <img> element bounds. */
function rectToNormalisedCrop(
rect: DragRect,
imgEl: HTMLImageElement,
): { x: number; y: number; w: number; h: number } | null {
const imgBounds = imgEl.getBoundingClientRect();
if (imgBounds.width === 0 || imgBounds.height === 0) return null;
// rect coords are already in viewport space (client coords)
const rawX = Math.min(rect.startX, rect.endX);
const rawY = Math.min(rect.startY, rect.endY);
const rawW = Math.abs(rect.endX - rect.startX);
const rawH = Math.abs(rect.endY - rect.startY);
// Clamp to image bounds
const clampedX = Math.max(rawX, imgBounds.left);
const clampedY = Math.max(rawY, imgBounds.top);
const clampedRight = Math.min(rawX + rawW, imgBounds.right);
const clampedBottom = Math.min(rawY + rawH, imgBounds.bottom);
const croppedW = clampedRight - clampedX;
const croppedH = clampedBottom - clampedY;
if (croppedW <= 0 || croppedH <= 0) return null;
// Normalize by the CSS transform scale — getBoundingClientRect already returns
// the scaled (on-screen) size, so we normalize directly against that.
return {
x: (clampedX - imgBounds.left) / imgBounds.width,
y: (clampedY - imgBounds.top) / imgBounds.height,
w: croppedW / imgBounds.width,
h: croppedH / imgBounds.height,
};
}
/** Minimum selection size as a fraction of the viewport container dimension. */
const MIN_SELECTION_FRACTION = 0.02;
export function Lightbox() { export function Lightbox() {
const selectedImage = useGalleryStore((state) => state.selectedImage); const selectedImage = useGalleryStore((state) => state.selectedImage);
const closeImage = useGalleryStore((state) => state.closeImage); const closeImage = useGalleryStore((state) => state.closeImage);
const images = useGalleryStore((state) => state.images); const images = useGalleryStore((state) => state.images);
const openImage = useGalleryStore((state) => state.openImage); const openImage = useGalleryStore((state) => state.openImage);
const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages); const loadSimilarImages = useGalleryStore((state) => state.loadSimilarImages);
const loadSimilarByRegion = useGalleryStore((state) => state.loadSimilarByRegion);
const similarScope = useGalleryStore((state) => state.similarScope);
const updateImageDetails = useGalleryStore((state) => state.updateImageDetails); const updateImageDetails = useGalleryStore((state) => state.updateImageDetails);
const getImageTags = useGalleryStore((state) => state.getImageTags); const getImageTags = useGalleryStore((state) => state.getImageTags);
const addUserTag = useGalleryStore((state) => state.addUserTag); const addUserTag = useGalleryStore((state) => state.addUserTag);
const removeTag = useGalleryStore((state) => state.removeTag); const removeTag = useGalleryStore((state) => state.removeTag);
const taggerModelStatus = useGalleryStore((state) => state.taggerModelStatus); const taggerModelStatus = useGalleryStore((state) => state.taggerModelStatus);
const queueTaggingForImage = useGalleryStore((state) => state.queueTaggingForImage); const queueTaggingForImage = useGalleryStore((state) => state.queueTaggingForImage);
const [zoom, setZoom] = useState(1); const [zoom, setZoom] = useState(1);
const [imageTags, setImageTags] = useState<ImageTag[]>([]); const [imageTags, setImageTags] = useState<ImageTag[]>([]);
const [tagInput, setTagInput] = useState(""); const [tagInput, setTagInput] = useState("");
const [tagAdding, setTagAdding] = useState(false); const [tagAdding, setTagAdding] = useState(false);
const [tagsExpanded, setTagsExpanded] = useState(false); const [tagsExpanded, setTagsExpanded] = useState(false);
const [taggingQueued, setTaggingQueued] = useState(false); const [taggingQueued, setTaggingQueued] = useState(false);
// Region selection state
const [regionSelectMode, setRegionSelectMode] = useState(false);
const [isDragging, setIsDragging] = useState(false);
const [dragRect, setDragRect] = useState<DragRect | null>(null);
const [regionSearching, setRegionSearching] = useState(false);
const imageViewportRef = useRef<HTMLDivElement>(null); const imageViewportRef = useRef<HTMLDivElement>(null);
const imgRef = useRef<HTMLImageElement>(null);
const currentIndex = selectedImage ? images.findIndex((image) => image.id === selectedImage.id) : -1; const currentIndex = selectedImage ? images.findIndex((image) => image.id === selectedImage.id) : -1;
const canFindSimilar = selectedImage?.embedding_status === "ready"; const canFindSimilar = selectedImage?.embedding_status === "ready";
const canSearchRegion = canFindSimilar && selectedImage?.media_kind === "image";
const goPrev = useCallback(() => { const goPrev = useCallback(() => {
if (currentIndex > 0) openImage(images[currentIndex - 1]); if (currentIndex > 0) openImage(images[currentIndex - 1]);
@@ -89,13 +157,21 @@ export function Lightbox() {
if (currentIndex >= 0 && currentIndex < images.length - 1) openImage(images[currentIndex + 1]); if (currentIndex >= 0 && currentIndex < images.length - 1) openImage(images[currentIndex + 1]);
}, [currentIndex, images, openImage]); }, [currentIndex, images, openImage]);
const exitRegionMode = useCallback(() => {
setRegionSelectMode(false);
setIsDragging(false);
setDragRect(null);
}, []);
useEffect(() => { useEffect(() => {
setZoom(1); setZoom(1);
setImageTags([]); setImageTags([]);
setTagInput(""); setTagInput("");
setTagsExpanded(false); setTagsExpanded(false);
setTaggingQueued(false); setTaggingQueued(false);
}, [selectedImage?.id]); exitRegionMode();
setRegionSearching(false);
}, [selectedImage?.id, exitRegionMode]);
useEffect(() => { useEffect(() => {
if (!selectedImage) return; if (!selectedImage) return;
@@ -114,6 +190,7 @@ export function Lightbox() {
if (!viewport || !selectedImage || selectedImage.media_kind !== "image") return; if (!viewport || !selectedImage || selectedImage.media_kind !== "image") return;
const handleWheel = (event: WheelEvent) => { const handleWheel = (event: WheelEvent) => {
if (regionSelectMode) return; // don't zoom during selection
if (!event.ctrlKey && Math.abs(event.deltaY) < Math.abs(event.deltaX)) return; if (!event.ctrlKey && Math.abs(event.deltaY) < Math.abs(event.deltaX)) return;
event.preventDefault(); event.preventDefault();
setZoom((value) => { setZoom((value) => {
@@ -124,12 +201,19 @@ export function Lightbox() {
viewport.addEventListener("wheel", handleWheel, { passive: false }); viewport.addEventListener("wheel", handleWheel, { passive: false });
return () => viewport.removeEventListener("wheel", handleWheel); return () => viewport.removeEventListener("wheel", handleWheel);
}, [selectedImage]); }, [selectedImage, regionSelectMode]);
useEffect(() => { useEffect(() => {
const handler = (event: KeyboardEvent) => { const handler = (event: KeyboardEvent) => {
if (!selectedImage) return; if (!selectedImage) return;
if (event.key === "Escape") closeImage(); if (event.key === "Escape") {
if (regionSelectMode) {
exitRegionMode();
} else {
closeImage();
}
}
if (regionSelectMode) return; // block nav keys during selection
if (event.key === "ArrowLeft") goPrev(); if (event.key === "ArrowLeft") goPrev();
if (event.key === "ArrowRight") goNext(); if (event.key === "ArrowRight") goNext();
if (event.key === "+" || event.key === "=") setZoom((value) => Math.min(3, value + 0.25)); if (event.key === "+" || event.key === "=") setZoom((value) => Math.min(3, value + 0.25));
@@ -138,7 +222,75 @@ export function Lightbox() {
window.addEventListener("keydown", handler); window.addEventListener("keydown", handler);
return () => window.removeEventListener("keydown", handler); return () => window.removeEventListener("keydown", handler);
}, [selectedImage, closeImage, goPrev, goNext]); }, [selectedImage, closeImage, goPrev, goNext, regionSelectMode, exitRegionMode]);
// ── Region selection pointer handlers ───────────────────────────────────────
const handleRegionPointerDown = useCallback(
(event: React.PointerEvent<HTMLDivElement>) => {
if (!regionSelectMode) return;
event.preventDefault();
event.currentTarget.setPointerCapture(event.pointerId);
setIsDragging(true);
setDragRect({
startX: event.clientX,
startY: event.clientY,
endX: event.clientX,
endY: event.clientY,
});
},
[regionSelectMode],
);
const handleRegionPointerMove = useCallback(
(event: React.PointerEvent<HTMLDivElement>) => {
if (!isDragging) return;
setDragRect((prev) =>
prev ? { ...prev, endX: event.clientX, endY: event.clientY } : null,
);
},
[isDragging],
);
const handleRegionPointerUp = useCallback(
(event: React.PointerEvent<HTMLDivElement>) => {
if (!isDragging || !dragRect || !selectedImage || !imgRef.current) {
setIsDragging(false);
return;
}
event.currentTarget.releasePointerCapture(event.pointerId);
const finalRect: DragRect = { ...dragRect, endX: event.clientX, endY: event.clientY };
const crop = rectToNormalisedCrop(finalRect, imgRef.current);
setIsDragging(false);
setDragRect(null);
// Ignore tiny accidental clicks
const containerBounds = imageViewportRef.current?.getBoundingClientRect();
const containerSize = containerBounds
? Math.min(containerBounds.width, containerBounds.height)
: 500;
const selW = Math.abs(finalRect.endX - finalRect.startX);
const selH = Math.abs(finalRect.endY - finalRect.startY);
if (!crop || selW < containerSize * MIN_SELECTION_FRACTION || selH < containerSize * MIN_SELECTION_FRACTION) {
exitRegionMode();
return;
}
exitRegionMode();
setRegionSearching(true);
const folderId =
similarScope === "current_folder" ? selectedImage.folder_id : null;
void loadSimilarByRegion(selectedImage.id, crop, folderId, selectedImage.folder_id)
.finally(() => setRegionSearching(false));
},
[isDragging, dragRect, selectedImage, similarScope, loadSimilarByRegion, exitRegionMode],
);
// Build the CSS rect for the selection overlay (viewport-relative)
const selectionOverlay =
isDragging && dragRect ? normaliseRect(dragRect) : null;
return ( return (
<AnimatePresence> <AnimatePresence>
@@ -150,11 +302,11 @@ export function Lightbox() {
animate={{ opacity: 1 }} animate={{ opacity: 1 }}
exit={{ opacity: 0 }} exit={{ opacity: 0 }}
transition={{ duration: 0.15 }} transition={{ duration: 0.15 }}
onClick={closeImage} onClick={regionSelectMode ? undefined : closeImage}
> >
<button <button
className="absolute left-4 top-1/2 z-10 -translate-y-1/2 rounded-full bg-white/10 p-3 text-white transition-colors hover:bg-white/20 disabled:opacity-20" className="absolute left-4 top-1/2 z-10 -translate-y-1/2 rounded-full bg-white/10 p-3 text-white transition-colors hover:bg-white/20 disabled:opacity-20"
disabled={currentIndex <= 0} disabled={currentIndex <= 0 || regionSelectMode}
onClick={(event) => { onClick={(event) => {
event.stopPropagation(); event.stopPropagation();
goPrev(); goPrev();
@@ -169,8 +321,38 @@ export function Lightbox() {
<div className="flex flex-1 overflow-hidden"> <div className="flex flex-1 overflow-hidden">
<div <div
ref={imageViewportRef} ref={imageViewportRef}
className="group relative flex flex-1 items-center justify-center overflow-auto p-10" className={`group relative flex flex-1 items-center justify-center overflow-auto p-10 ${
regionSelectMode ? "cursor-crosshair select-none" : ""
}`}
onPointerDown={handleRegionPointerDown}
onPointerMove={handleRegionPointerMove}
onPointerUp={handleRegionPointerUp}
> >
{/* Region selection mode hint */}
{regionSelectMode && (
<div className="pointer-events-none absolute inset-x-0 top-4 z-20 flex justify-center">
<div className="flex items-center gap-2 rounded-full border border-white/15 bg-black/70 px-4 py-2 text-xs text-gray-300 backdrop-blur">
<svg className="h-3.5 w-3.5 text-violet-400" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 8V4m0 0h4M4 4l5 5m11-1V4m0 0h-4m4 0l-5 5M4 16v4m0 0h4m-4 0l5-5m11 5l-5-5m5 5v-4m0 4h-4" />
</svg>
Draw a region to search <kbd className="ml-1 rounded border border-white/20 bg-white/10 px-1.5 py-0.5 font-mono text-[10px]">Esc</kbd> to cancel
</div>
</div>
)}
{/* Selection rectangle overlay */}
{selectionOverlay && selectionOverlay.width > 4 && selectionOverlay.height > 4 && (
<div
className="pointer-events-none fixed z-30 rounded border-2 border-violet-400 bg-violet-400/15 shadow-[0_0_0_9999px_rgba(0,0,0,0.35)]"
style={{
left: selectionOverlay.left,
top: selectionOverlay.top,
width: selectionOverlay.width,
height: selectionOverlay.height,
}}
/>
)}
<AnimatePresence mode="wait"> <AnimatePresence mode="wait">
<motion.div <motion.div
key={selectedImage.id} key={selectedImage.id}
@@ -190,6 +372,7 @@ export function Lightbox() {
) : ( ) : (
<> <>
<img <img
ref={imgRef}
src={convertFileSrc(selectedImage.path)} src={convertFileSrc(selectedImage.path)}
alt={selectedImage.filename} alt={selectedImage.filename}
className="max-w-full rounded-2xl shadow-2xl" className="max-w-full rounded-2xl shadow-2xl"
@@ -197,25 +380,29 @@ export function Lightbox() {
maxHeight: "calc(100vh - 10rem)", maxHeight: "calc(100vh - 10rem)",
transform: `scale(${zoom})`, transform: `scale(${zoom})`,
transformOrigin: "center center", transformOrigin: "center center",
// Slightly dim the image while in region select mode
...(regionSelectMode ? { opacity: 0.85 } : {}),
}} }}
/> />
<div className="pointer-events-none absolute right-6 top-6 opacity-0 transition-opacity group-hover:opacity-100"> {!regionSelectMode && (
<div className="pointer-events-auto flex items-center gap-1 rounded-full border border-white/10 bg-black/55 px-2 py-1 backdrop-blur"> <div className="pointer-events-none absolute right-6 top-6 opacity-0 transition-opacity group-hover:opacity-100">
<button <div className="pointer-events-auto flex items-center gap-1 rounded-full border border-white/10 bg-black/55 px-2 py-1 backdrop-blur">
className="px-2 text-sm text-gray-300 hover:text-white" <button
onClick={() => setZoom((value) => Math.max(0.5, value - 0.25))} className="px-2 text-sm text-gray-300 hover:text-white"
> onClick={() => setZoom((value) => Math.max(0.5, value - 0.25))}
- >
</button> -
<span className="min-w-14 text-center text-xs text-gray-300">{Math.round(zoom * 100)}%</span> </button>
<button <span className="min-w-14 text-center text-xs text-gray-300">{Math.round(zoom * 100)}%</span>
className="px-2 text-sm text-gray-300 hover:text-white" <button
onClick={() => setZoom((value) => Math.min(4, value + 0.25))} className="px-2 text-sm text-gray-300 hover:text-white"
> onClick={() => setZoom((value) => Math.min(4, value + 0.25))}
+ >
</button> +
</button>
</div>
</div> </div>
</div> )}
</> </>
)} )}
</motion.div> </motion.div>
@@ -246,7 +433,7 @@ export function Lightbox() {
}`} }`}
onClick={() => { onClick={() => {
if (!canFindSimilar) return; if (!canFindSimilar) return;
void loadSimilarImages(selectedImage.id); void loadSimilarImages(selectedImage.id, similarScope === "current_folder" ? selectedImage.folder_id : null, true, selectedImage.folder_id);
}} }}
disabled={!canFindSimilar} disabled={!canFindSimilar}
> >
@@ -260,6 +447,53 @@ export function Lightbox() {
</button> </button>
</div> </div>
{/* Search region button row */}
{canSearchRegion && (
<div className="shrink-0 px-5 pb-3">
<button
className={`w-full rounded-lg border px-3 py-2 text-xs transition-colors ${
regionSelectMode
? "border-violet-400/40 bg-violet-500/15 text-violet-300 hover:bg-violet-500/20"
: regionSearching
? "border-white/5 bg-white/[0.03] text-gray-500 cursor-not-allowed"
: "border-white/10 bg-white/5 text-gray-300 hover:bg-white/10 hover:text-white"
}`}
onClick={() => {
if (regionSearching) return;
setRegionSelectMode((prev) => !prev);
setDragRect(null);
setIsDragging(false);
}}
disabled={regionSearching}
title={regionSelectMode ? "Cancel region selection" : "Draw a region on the image to search for similar content"}
>
{regionSearching ? (
<span className="flex items-center justify-center gap-1.5">
<svg className="h-3 w-3 animate-spin" fill="none" viewBox="0 0 24 24">
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4" />
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8v8H4z" />
</svg>
Searching region
</span>
) : regionSelectMode ? (
<span className="flex items-center justify-center gap-1.5">
<svg className="h-3 w-3" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</svg>
Cancel selection
</span>
) : (
<span className="flex items-center justify-center gap-1.5">
<svg className="h-3 w-3" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 8V4m0 0h4M4 4l5 5m11-1V4m0 0h-4m4 0l-5 5M4 16v4m0 0h4m-4 0l5-5m11 5l-5-5m5 5v-4m0 4h-4" />
</svg>
Search within image
</span>
)}
</button>
</div>
)}
<div className="min-h-0 flex-1 overflow-y-auto px-5 pb-4 space-y-4 text-sm"> <div className="min-h-0 flex-1 overflow-y-auto px-5 pb-4 space-y-4 text-sm">
<div> <div>
<p className="mb-1 text-xs uppercase tracking-wider text-gray-500">Rating</p> <p className="mb-1 text-xs uppercase tracking-wider text-gray-500">Rating</p>
@@ -471,7 +705,7 @@ export function Lightbox() {
<button <button
className="absolute right-80 top-1/2 z-10 -translate-y-1/2 rounded-full bg-white/10 p-3 text-white transition-colors hover:bg-white/20 disabled:opacity-20" className="absolute right-80 top-1/2 z-10 -translate-y-1/2 rounded-full bg-white/10 p-3 text-white transition-colors hover:bg-white/20 disabled:opacity-20"
disabled={currentIndex >= images.length - 1} disabled={currentIndex >= images.length - 1 || regionSelectMode}
onClick={(event) => { onClick={(event) => {
event.stopPropagation(); event.stopPropagation();
goNext(); goNext();
+443 -337
View File
@@ -1,37 +1,87 @@
import { useEffect, useState } from "react"; import { useEffect, useMemo, useState } from "react";
import { TaggerAcceleration, useGalleryStore } from "../store"; import { TaggerAcceleration, TaggingQueueScope, useGalleryStore } from "../store";
type SettingsSection = "tagging" | "library" | "display" | "storage"; type SettingsSection = "workspace" | "workers";
const SECTIONS: { id: SettingsSection; label: string; detail: string }[] = [ const SECTIONS: { id: SettingsSection; label: string; detail: string }[] = [
{ id: "tagging", label: "AI Tagging", detail: "WD tagger model" }, { id: "workspace", label: "AI Workspace", detail: "Tagging models and queue targets" },
{ id: "library", label: "Library", detail: "Indexing and scanning" }, { id: "workers", label: "Workers", detail: "Queue activity and background processing" },
{ id: "display", label: "Display", detail: "Gallery preferences" },
{ id: "storage", label: "Storage", detail: "Cache and model files" },
]; ];
function StatusPill({ children, tone }: { children: React.ReactNode; tone: "ready" | "muted" | "busy" }) { function StatusPill({ children, tone }: { children: React.ReactNode; tone: "ready" | "muted" | "busy" }) {
const className = const className =
tone === "ready" tone === "ready"
? "border-emerald-400/25 bg-emerald-500/10 text-emerald-300" ? "border-emerald-400/25 bg-emerald-500/10 text-emerald-300"
: tone === "busy" : tone === "busy"
? "border-sky-400/25 bg-sky-500/10 text-sky-300" ? "border-sky-400/25 bg-sky-500/10 text-sky-300"
: "border-white/10 bg-white/[0.04] text-gray-500"; : "border-white/10 bg-white/[0.04] text-gray-500";
return <span className={`inline-flex rounded-md border px-2 py-1 text-[11px] font-medium ${className}`}>{children}</span>;
}
function SectionShell({ eyebrow, title, description, children }: {
eyebrow: string;
title: string;
description?: string;
children: React.ReactNode;
}) {
return ( return (
<span className={`inline-flex rounded-md border px-2 py-1 text-[11px] font-medium ${className}`}> <section>
{children} <p className="text-[10px] font-semibold uppercase tracking-[0.18em] text-gray-600">{eyebrow}</p>
</span> <h3 className="mt-1 text-lg font-semibold text-white">{title}</h3>
{description ? <p className="mt-2 max-w-2xl text-sm leading-relaxed text-gray-500">{description}</p> : null}
<div className="mt-5 space-y-4">{children}</div>
</section>
); );
} }
function TaggerAccelerationButton({ function SettingsCard({ title, description, children }: { title: string; description?: string; children: React.ReactNode }) {
acceleration, return (
current, <div className="rounded-2xl border border-white/[0.07] bg-white/[0.03] p-5">
onSelect, <div className="flex flex-col gap-1 border-b border-white/[0.07] pb-4">
children, <p className="text-sm font-medium text-white">{title}</p>
}: { {description ? <p className="text-xs leading-relaxed text-gray-500">{description}</p> : null}
</div>
<div className="pt-4">{children}</div>
</div>
);
}
function SettingsRow({ title, description, children }: { title: string; description: string; children: React.ReactNode }) {
return (
<div className="flex items-start justify-between gap-5 border-b border-white/[0.07] py-4 last:border-b-0 first:pt-0 last:pb-0">
<div className="min-w-0">
<p className="text-sm font-medium text-white">{title}</p>
<p className="mt-1 max-w-md text-xs leading-relaxed text-gray-500">{description}</p>
</div>
<div className="shrink-0">{children}</div>
</div>
);
}
function ScopeButton({ scope, current, onSelect, children }: {
scope: TaggingQueueScope;
current: TaggingQueueScope;
onSelect: (scope: TaggingQueueScope) => void;
children: React.ReactNode;
}) {
const active = scope === current;
return (
<button
type="button"
className={`rounded-md border px-3 py-1.5 text-xs transition-colors ${
active
? "border-emerald-400/35 bg-emerald-500/15 text-emerald-200"
: "border-white/10 bg-white/[0.045] text-gray-500 hover:bg-white/[0.075] hover:text-gray-200"
}`}
onClick={() => onSelect(scope)}
>
{children}
</button>
);
}
function TaggerAccelerationButton({ acceleration, current, onSelect, children }: {
acceleration: TaggerAcceleration; acceleration: TaggerAcceleration;
current: TaggerAcceleration; current: TaggerAcceleration;
onSelect: (acceleration: TaggerAcceleration) => void; onSelect: (acceleration: TaggerAcceleration) => void;
@@ -53,63 +103,32 @@ function TaggerAccelerationButton({
); );
} }
function SettingsRow({
title,
description,
children,
}: {
title: string;
description: string;
children: React.ReactNode;
}) {
return (
<div className="flex items-start justify-between gap-5 border-b border-white/[0.07] py-4 last:border-b-0">
<div className="min-w-0">
<p className="text-sm font-medium text-white">{title}</p>
<p className="mt-1 max-w-md text-xs leading-relaxed text-gray-500">{description}</p>
</div>
<div className="shrink-0">{children}</div>
</div>
);
}
function SectionShell({
eyebrow,
title,
children,
}: {
eyebrow: string;
title: string;
children: React.ReactNode;
}) {
return (
<div>
<p className="text-[10px] font-semibold uppercase tracking-[0.18em] text-gray-600">{eyebrow}</p>
<h3 className="mt-1 text-lg font-semibold text-white">{title}</h3>
<div className="mt-5 border-t border-white/[0.08]">{children}</div>
</div>
);
}
export function SettingsModal() { export function SettingsModal() {
const [activeSection, setActiveSection] = useState<SettingsSection>("tagging"); const [activeSection, setActiveSection] = useState<SettingsSection>("workspace");
const [taggerQueueStatus, setTaggerQueueStatus] = useState<string | null>(null); const [taggerQueueStatus, setTaggerQueueStatus] = useState<string | null>(null);
const [taggerQueueing, setTaggerQueueing] = useState(false); const [taggerQueueing, setTaggerQueueing] = useState(false);
const [taggerClearing, setTaggerClearing] = useState(false); const [taggerClearing, setTaggerClearing] = useState(false);
const [taggerAccelerationSaving, setTaggerAccelerationSaving] = useState(false); const [taggerAccelerationSaving, setTaggerAccelerationSaving] = useState(false);
const [taggerThresholdDraft, setTaggerThresholdDraft] = useState<string | null>(null); const [taggerThresholdDraft, setTaggerThresholdDraft] = useState<string | null>(null);
const [taggerThresholdSaving, setTaggerThresholdSaving] = useState(false); const [taggerThresholdSaving, setTaggerThresholdSaving] = useState(false);
const [taggerBatchSizeDraft, setTaggerBatchSizeDraft] = useState<string | null>(null);
const [taggerBatchSizeSaving, setTaggerBatchSizeSaving] = useState(false);
const settingsOpen = useGalleryStore((state) => state.settingsOpen); const settingsOpen = useGalleryStore((state) => state.settingsOpen);
const setSettingsOpen = useGalleryStore((state) => state.setSettingsOpen); const setSettingsOpen = useGalleryStore((state) => state.setSettingsOpen);
const selectedFolderId = useGalleryStore((state) => state.selectedFolderId);
const folders = useGalleryStore((state) => state.folders); const folders = useGalleryStore((state) => state.folders);
const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress);
const taggingQueueScope = useGalleryStore((state) => state.taggingQueueScope);
const taggingQueueFolderIds = useGalleryStore((state) => state.taggingQueueFolderIds);
const setTaggingQueueScope = useGalleryStore((state) => state.setTaggingQueueScope);
const toggleTaggingQueueFolder = useGalleryStore((state) => state.toggleTaggingQueueFolder);
const setTaggingQueueFolderIds = useGalleryStore((state) => state.setTaggingQueueFolderIds);
const taggerModelStatus = useGalleryStore((state) => state.taggerModelStatus); const taggerModelStatus = useGalleryStore((state) => state.taggerModelStatus);
const taggerModelPreparing = useGalleryStore((state) => state.taggerModelPreparing); const taggerModelPreparing = useGalleryStore((state) => state.taggerModelPreparing);
const taggerModelProgress = useGalleryStore((state) => state.taggerModelProgress); const taggerModelProgress = useGalleryStore((state) => state.taggerModelProgress);
const taggerModelError = useGalleryStore((state) => state.taggerModelError); const taggerModelError = useGalleryStore((state) => state.taggerModelError);
const taggerAcceleration = useGalleryStore((state) => state.taggerAcceleration); const taggerAcceleration = useGalleryStore((state) => state.taggerAcceleration);
const taggerThreshold = useGalleryStore((state) => state.taggerThreshold); const taggerThreshold = useGalleryStore((state) => state.taggerThreshold);
const taggerBatchSize = useGalleryStore((state) => state.taggerBatchSize);
const taggerRuntimeProbe = useGalleryStore((state) => state.taggerRuntimeProbe); const taggerRuntimeProbe = useGalleryStore((state) => state.taggerRuntimeProbe);
const taggerRuntimeChecking = useGalleryStore((state) => state.taggerRuntimeChecking); const taggerRuntimeChecking = useGalleryStore((state) => state.taggerRuntimeChecking);
const loadTaggerModelStatus = useGalleryStore((state) => state.loadTaggerModelStatus); const loadTaggerModelStatus = useGalleryStore((state) => state.loadTaggerModelStatus);
@@ -119,49 +138,123 @@ export function SettingsModal() {
const setTaggerAcceleration = useGalleryStore((state) => state.setTaggerAcceleration); const setTaggerAcceleration = useGalleryStore((state) => state.setTaggerAcceleration);
const loadTaggerThreshold = useGalleryStore((state) => state.loadTaggerThreshold); const loadTaggerThreshold = useGalleryStore((state) => state.loadTaggerThreshold);
const setTaggerThreshold = useGalleryStore((state) => state.setTaggerThreshold); const setTaggerThreshold = useGalleryStore((state) => state.setTaggerThreshold);
const loadTaggerBatchSize = useGalleryStore((state) => state.loadTaggerBatchSize);
const setTaggerBatchSize = useGalleryStore((state) => state.setTaggerBatchSize);
const probeTaggerRuntime = useGalleryStore((state) => state.probeTaggerRuntime); const probeTaggerRuntime = useGalleryStore((state) => state.probeTaggerRuntime);
const queueTaggingJobs = useGalleryStore((state) => state.queueTaggingJobs); const queueTaggingJobs = useGalleryStore((state) => state.queueTaggingJobs);
const queueTaggingJobsForFolders = useGalleryStore((state) => state.queueTaggingJobsForFolders);
const clearTaggingJobs = useGalleryStore((state) => state.clearTaggingJobs); const clearTaggingJobs = useGalleryStore((state) => state.clearTaggingJobs);
const clearTaggingJobsForFolders = useGalleryStore((state) => state.clearTaggingJobsForFolders);
useEffect(() => { useEffect(() => {
if (!settingsOpen) return; if (!settingsOpen) return;
void loadTaggerModelStatus(); void loadTaggerModelStatus();
void loadTaggerAcceleration(); void loadTaggerAcceleration();
void loadTaggerThreshold(); void loadTaggerThreshold();
void loadTaggerBatchSize();
const handleKeyDown = (event: KeyboardEvent) => { const handleKeyDown = (event: KeyboardEvent) => {
if (event.key === "Escape") setSettingsOpen(false); if (event.key === "Escape") setSettingsOpen(false);
}; };
window.addEventListener("keydown", handleKeyDown); window.addEventListener("keydown", handleKeyDown);
return () => window.removeEventListener("keydown", handleKeyDown); return () => window.removeEventListener("keydown", handleKeyDown);
}, [settingsOpen, loadTaggerModelStatus, loadTaggerAcceleration, loadTaggerThreshold, setSettingsOpen]); }, [settingsOpen, loadTaggerModelStatus, loadTaggerAcceleration, loadTaggerThreshold, loadTaggerBatchSize, setSettingsOpen]);
const selectedFolders = useMemo(
() => folders.filter((folder) => taggingQueueFolderIds.includes(folder.id)),
[folders, taggingQueueFolderIds],
);
const totalQueuedJobs = useMemo(
() => Object.values(mediaJobProgress).reduce((sum, progress) => sum + (progress?.tagging_pending ?? 0), 0),
[mediaJobProgress],
);
if (!settingsOpen) return null; if (!settingsOpen) return null;
const selectedFolder = folders.find((folder) => folder.id === selectedFolderId); const taggerReady = taggerModelStatus?.ready ?? false;
const scopeLabel = selectedFolder ? selectedFolder.name : "all libraries"; const queueScopeLabel =
taggingQueueScope === "all"
? "all media"
: selectedFolders.length > 0
? `${selectedFolders.length} selected folder${selectedFolders.length === 1 ? "" : "s"}`
: "no folders selected";
const thresholdDisplay = taggerThresholdDraft ?? String(taggerThreshold);
const batchSizeDisplay = taggerBatchSizeDraft ?? String(taggerBatchSize);
const taggerDownloadLabel = taggerModelProgress
? `Downloading ${taggerModelProgress.completed_files}/${taggerModelProgress.total_files}`
: taggerModelPreparing
? "Preparing WD Tagger..."
: taggerReady
? "Installed"
: "Install model";
const taggerDownloadPercent = taggerModelProgress
? Math.round((taggerModelProgress.completed_files / Math.max(taggerModelProgress.total_files, 1)) * 100)
: 0;
const runQueueAction = (action: "queue" | "clear") => {
const selectedIds = taggingQueueFolderIds;
const perform =
taggingQueueScope === "all"
? action === "queue"
? queueTaggingJobs(null)
: clearTaggingJobs(null)
: selectedIds.length > 0
? action === "queue"
? queueTaggingJobsForFolders(selectedIds)
: clearTaggingJobsForFolders(selectedIds)
: Promise.resolve(0);
if (action === "queue") {
setTaggerQueueing(true);
} else {
setTaggerClearing(true);
}
setTaggerQueueStatus(null);
void perform
.then((count) => {
if (taggingQueueScope === "selected" && selectedIds.length === 0) {
setTaggerQueueStatus("Choose at least one folder before running tagging jobs.");
return;
}
setTaggerQueueStatus(
count === 0
? action === "queue"
? "No missing tags found for the current target."
: "No queued tagging jobs to clear for the current target."
: action === "queue"
? `Queued ${count.toLocaleString()} image${count === 1 ? "" : "s"} for tagging.`
: `Cleared ${count.toLocaleString()} queued tagging job${count === 1 ? "" : "s"}.`,
);
})
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => {
if (action === "queue") {
setTaggerQueueing(false);
} else {
setTaggerClearing(false);
}
});
};
return ( return (
<div <div className="fixed inset-0 z-[60] flex items-center justify-center bg-black/65 px-6 backdrop-blur-sm" onClick={() => setSettingsOpen(false)}>
className="fixed inset-0 z-[60] flex items-center justify-center bg-black/65 px-6 backdrop-blur-sm"
onClick={() => setSettingsOpen(false)}
>
<div <div
className="flex h-[min(680px,calc(100vh-56px))] w-full max-w-4xl overflow-hidden rounded-lg border border-white/10 bg-[#07080f] shadow-2xl shadow-black/60" className="flex h-[min(760px,calc(100vh-56px))] w-full max-w-5xl overflow-hidden rounded-lg border border-white/10 bg-[#07080f] shadow-2xl shadow-black/60"
onClick={(event) => event.stopPropagation()} onClick={(event) => event.stopPropagation()}
> >
<aside className="flex w-56 shrink-0 flex-col border-r border-white/[0.07] bg-white/[0.025]"> <aside className="flex w-64 shrink-0 flex-col border-r border-white/[0.07] bg-white/[0.025]">
<div className="border-b border-white/[0.07] px-5 py-5"> <div className="border-b border-white/[0.07] px-5 py-5">
<p className="text-base font-semibold text-white">Settings</p> <p className="text-base font-semibold text-white">Settings</p>
<p className="mt-1 text-xs text-gray-600">Phokus preferences</p> <p className="mt-1 text-xs text-gray-600">Operational controls for AI workflows</p>
</div> </div>
<div className="flex-1 space-y-1 overflow-y-auto p-2"> <div className="flex-1 space-y-1 overflow-y-auto p-2">
{SECTIONS.map((section) => ( {SECTIONS.map((section) => (
<button <button
key={section.id} key={section.id}
className={`w-full rounded-md px-3 py-2.5 text-left transition-colors ${ className={`w-full rounded-md px-3 py-2.5 text-left transition-colors ${
activeSection === section.id activeSection === section.id ? "bg-white/10 text-white" : "text-gray-500 hover:bg-white/[0.055] hover:text-gray-200"
? "bg-white/10 text-white"
: "text-gray-500 hover:bg-white/[0.055] hover:text-gray-200"
}`} }`}
onClick={() => setActiveSection(section.id)} onClick={() => setActiveSection(section.id)}
> >
@@ -173,7 +266,11 @@ export function SettingsModal() {
</aside> </aside>
<main className="flex min-w-0 flex-1 flex-col"> <main className="flex min-w-0 flex-1 flex-col">
<div className="flex h-14 shrink-0 items-center justify-end border-b border-white/[0.07] px-6"> <div className="flex h-14 shrink-0 items-center justify-between border-b border-white/[0.07] px-6">
<div>
<p className="text-sm font-medium text-white">{activeSection === "workspace" ? "AI Workspace" : "Workers"}</p>
<p className="text-xs text-gray-600">{activeSection === "workspace" ? "Model setup and queue targets" : "Background processing status"}</p>
</div>
<button <button
className="rounded-md p-1.5 text-gray-500 transition-colors hover:bg-white/[0.06] hover:text-white" className="rounded-md p-1.5 text-gray-500 transition-colors hover:bg-white/[0.06] hover:text-white"
onClick={() => setSettingsOpen(false)} onClick={() => setSettingsOpen(false)}
@@ -186,274 +283,283 @@ export function SettingsModal() {
</div> </div>
<div className="flex-1 overflow-y-auto px-7 py-6"> <div className="flex-1 overflow-y-auto px-7 py-6">
{activeSection === "tagging" ? (() => { {activeSection === "workspace" ? (
const taggerReady = taggerModelStatus?.ready ?? false; <div className="space-y-8">
const taggerDownloadLabel = taggerModelProgress <SectionShell
? `Downloading ${taggerModelProgress.completed_files}/${taggerModelProgress.total_files}` eyebrow="AI Workspace"
: taggerModelPreparing title="Tagging"
? "Preparing WD Tagger..." >
: taggerReady <SettingsCard title="Tagging Models">
? "Downloaded" <div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
: "Download WD Tagger"; <div className="flex items-start justify-between gap-4">
const taggerDownloadPercent = taggerModelProgress <div>
? Math.round((taggerModelProgress.completed_files / Math.max(taggerModelProgress.total_files, 1)) * 100) <div className="flex items-center gap-2">
: 0; <p className="text-sm font-medium text-white">WD SwinV2 Tagger v3</p>
const thresholdDisplay = taggerThresholdDraft ?? String(taggerThreshold); <StatusPill tone={taggerReady ? "ready" : taggerModelPreparing ? "busy" : "muted"}>
return ( {taggerReady ? "Installed" : taggerModelPreparing ? "Preparing" : "Not installed"}
<SectionShell eyebrow="AI Tagging" title="WD SwinV2 Tagger v3"> </StatusPill>
<SettingsRow
title="WD Tagger model"
description={taggerReady ? "Stored locally and available offline." : "Download the model to enable automatic AI tagging."}
>
<div className="flex flex-col items-end gap-2">
<button
className="relative overflow-hidden rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void prepareTaggerModel()}
disabled={taggerModelPreparing || taggerReady}
>
{taggerModelProgress ? (
<span
className="absolute inset-y-0 left-0 bg-emerald-400/15 transition-[width] duration-200"
style={{ width: `${taggerDownloadPercent}%` }}
/>
) : null}
<span className="relative">{taggerDownloadLabel}</span>
</button>
{taggerReady ? (
<>
<button
className="rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void probeTaggerRuntime()}
disabled={taggerRuntimeChecking}
>
{taggerRuntimeChecking ? "Checking runtime..." : "Check runtime"}
</button>
<button
className="rounded-md border border-red-400/20 bg-red-500/10 px-3 py-1.5 text-xs text-red-200 transition-colors hover:bg-red-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void deleteTaggerModel()}
disabled={taggerModelPreparing}
>
Delete model files
</button>
</>
) : null}
</div>
</SettingsRow>
<SettingsRow
title="Tagger acceleration"
description="Use DirectML for GPU-accelerated tagging when available."
>
<div className="flex flex-col items-end gap-2">
<div className="flex rounded-lg border border-white/[0.07] bg-black/20 p-1">
<TaggerAccelerationButton
acceleration="auto"
current={taggerAcceleration}
onSelect={(acceleration) => {
setTaggerAccelerationSaving(true);
void setTaggerAcceleration(acceleration)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerAccelerationSaving(false));
}}
>
Auto
</TaggerAccelerationButton>
<TaggerAccelerationButton
acceleration="directml"
current={taggerAcceleration}
onSelect={(acceleration) => {
setTaggerAccelerationSaving(true);
void setTaggerAcceleration(acceleration)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerAccelerationSaving(false));
}}
>
DirectML
</TaggerAccelerationButton>
<TaggerAccelerationButton
acceleration="cpu"
current={taggerAcceleration}
onSelect={(acceleration) => {
setTaggerAccelerationSaving(true);
void setTaggerAcceleration(acceleration)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerAccelerationSaving(false));
}}
>
CPU
</TaggerAccelerationButton>
</div>
<p className="text-[11px] text-gray-600">
{taggerAccelerationSaving ? "Saving..." : `Current: ${taggerAcceleration}`}
</p>
</div>
</SettingsRow>
<SettingsRow
title="Confidence threshold"
description="Tags with confidence below this value are discarded. Lower = more tags, higher = fewer but more accurate."
>
<div className="flex flex-col items-end gap-2">
<div className="flex items-center gap-2">
<input
type="number"
min="0.05"
max="0.99"
step="0.05"
className="w-20 rounded-md border border-white/10 bg-white/5 px-2 py-1.5 text-xs text-white focus:border-white/20 focus:outline-none"
value={thresholdDisplay}
onChange={(e) => setTaggerThresholdDraft(e.target.value)}
onBlur={() => {
const val = parseFloat(thresholdDisplay);
if (!isNaN(val) && val >= 0.05 && val <= 0.99) {
setTaggerThresholdSaving(true);
void setTaggerThreshold(val)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => {
setTaggerThresholdDraft(null);
setTaggerThresholdSaving(false);
});
} else {
setTaggerThresholdDraft(null);
}
}}
/>
</div>
<p className="text-[11px] text-gray-600">
{taggerThresholdSaving ? "Saving..." : "Default: 0.35"}
</p>
</div>
</SettingsRow>
<SettingsRow
title="Tagging queue"
description={`Generate missing AI tags in ${scopeLabel}. Tags update as the background worker finishes each image.`}
>
<div className="flex flex-col items-end gap-2">
<button
className="rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => {
setTaggerQueueing(true);
setTaggerQueueStatus(null);
void queueTaggingJobs(selectedFolderId)
.then((queued) => {
setTaggerQueueStatus(
queued === 0
? "No missing tags found."
: `Queued ${queued.toLocaleString()} image${queued === 1 ? "" : "s"}.`,
);
})
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerQueueing(false));
}}
disabled={!taggerReady || taggerQueueing || taggerClearing}
>
{taggerQueueing
? "Queueing..."
: selectedFolder
? "Tag this library"
: "Tag all libraries"}
</button>
<button
className="rounded-md border border-amber-400/20 bg-amber-500/10 px-3 py-1.5 text-xs text-amber-200 transition-colors hover:bg-amber-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => {
setTaggerClearing(true);
setTaggerQueueStatus(null);
void clearTaggingJobs(selectedFolderId)
.then((cleared) => {
setTaggerQueueStatus(
cleared === 0
? "No queued tagging jobs to clear."
: `Cleared ${cleared.toLocaleString()} queued job${cleared === 1 ? "" : "s"}.`,
);
})
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerClearing(false));
}}
disabled={taggerQueueing || taggerClearing}
>
{taggerClearing
? "Clearing..."
: selectedFolder
? "Clear this queue"
: "Clear all queued tags"}
</button>
</div>
</SettingsRow>
<div className="py-4">
<p className="text-xs font-medium text-gray-400">Model location</p>
<p className="mt-2 break-all rounded-md border border-white/[0.07] bg-black/20 px-3 py-2 text-xs text-gray-600">
{taggerReady ? taggerModelStatus?.local_dir : "Not downloaded"}
</p>
{taggerModelProgress?.current_file ? (
<p className="mt-3 break-all text-xs text-gray-500">{taggerModelProgress.current_file}</p>
) : null}
{taggerModelError ? (
<p className="mt-3 text-xs text-amber-300">{taggerModelError}</p>
) : null}
{taggerQueueStatus ? (
<p className="mt-3 text-xs text-gray-500">{taggerQueueStatus}</p>
) : null}
{taggerRuntimeProbe ? (
<div className="mt-4 border-t border-white/[0.07] pt-4">
<div className="flex items-center justify-between gap-3">
<p className="text-xs font-medium text-gray-400">Runtime check</p>
<StatusPill tone="ready">Ready</StatusPill>
</div>
<p className="mt-2 text-xs text-gray-600">
Tagger acceleration: {taggerRuntimeProbe.acceleration}
</p>
<div className="mt-3 space-y-2">
<div className="rounded-md border border-white/[0.07] bg-black/20 px-3 py-2">
<p className="break-all text-xs text-gray-400">{taggerRuntimeProbe.session.file}</p>
<p className="mt-1 text-[11px] text-gray-600">
{taggerRuntimeProbe.session.inputs.length} input{taggerRuntimeProbe.session.inputs.length === 1 ? "" : "s"} · {taggerRuntimeProbe.session.outputs.join(", ")}
</p>
</div> </div>
<p className="mt-1 max-w-xl text-xs leading-relaxed text-gray-500">
Anime-focused vision model by SmilingWolf. Generates booru-style tags with configurable confidence thresholds.
</p>
</div>
<div className="flex flex-col items-end gap-2">
<button
className="relative overflow-hidden rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void prepareTaggerModel()}
disabled={taggerModelPreparing || taggerReady}
>
{taggerModelProgress ? <span className="absolute inset-y-0 left-0 bg-emerald-400/15 transition-[width] duration-200" style={{ width: `${taggerDownloadPercent}%` }} /> : null}
<span className="relative">{taggerDownloadLabel}</span>
</button>
{taggerReady ? (
<>
<button
className="rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void probeTaggerRuntime()}
disabled={taggerRuntimeChecking}
>
{taggerRuntimeChecking ? "Checking runtime..." : "Check runtime"}
</button>
<button
className="rounded-md border border-red-400/20 bg-red-500/10 px-3 py-1.5 text-xs text-red-200 transition-colors hover:bg-red-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void deleteTaggerModel()}
disabled={taggerModelPreparing}
>
Delete model files
</button>
</>
) : null}
</div> </div>
</div> </div>
) : null}
</div> <div className="mt-4 space-y-4 border-t border-white/[0.07] pt-4">
<SettingsRow title="Tagger acceleration" description="Use DirectML when available, or fall back to CPU for reliability.">
<div className="flex flex-col items-end gap-2">
<div className="flex rounded-lg border border-white/[0.07] bg-black/20 p-1">
{(["auto", "directml", "cpu"] as const).map((acceleration) => (
<TaggerAccelerationButton
key={acceleration}
acceleration={acceleration}
current={taggerAcceleration}
onSelect={(nextAcceleration) => {
setTaggerAccelerationSaving(true);
void setTaggerAcceleration(nextAcceleration)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => setTaggerAccelerationSaving(false));
}}
>
{acceleration === "directml" ? "DirectML" : acceleration === "cpu" ? "CPU" : "Auto"}
</TaggerAccelerationButton>
))}
</div>
<p className="text-[11px] text-gray-600">{taggerAccelerationSaving ? "Saving..." : `Current: ${taggerAcceleration}`}</p>
</div>
</SettingsRow>
<SettingsRow title="Confidence threshold" description="Lower values keep more tags. Higher values are stricter and usually cleaner.">
<div className="flex flex-col items-end gap-2">
<input
type="number"
min="0.05"
max="0.99"
step="0.05"
className="w-20 rounded-md border border-white/10 bg-white/5 px-2 py-1.5 text-xs text-white focus:border-white/20 focus:outline-none"
value={thresholdDisplay}
onChange={(event) => setTaggerThresholdDraft(event.target.value)}
onBlur={() => {
const value = parseFloat(thresholdDisplay);
if (!isNaN(value) && value >= 0.05 && value <= 0.99) {
setTaggerThresholdSaving(true);
void setTaggerThreshold(value)
.catch((error) => setTaggerQueueStatus(String(error)))
.finally(() => {
setTaggerThresholdDraft(null);
setTaggerThresholdSaving(false);
});
} else {
setTaggerThresholdDraft(null);
}
}}
/>
<p className="text-[11px] text-gray-600">{taggerThresholdSaving ? "Saving..." : "Default: 0.35"}</p>
</div>
</SettingsRow>
<SettingsRow title="Tagging batch size" description="Number of images processed concurrently during tag generation.">
<div className="flex flex-col items-end gap-2">
<input
type="number"
min="1"
max="100"
step="1"
className="w-20 rounded-md border border-white/10 bg-white/5 px-2 py-1.5 text-xs text-white focus:border-white/20 focus:outline-none"
value={batchSizeDisplay}
onChange={(event) => setTaggerBatchSizeDraft(event.target.value)}
onBlur={() => {
const value = parseInt(batchSizeDisplay, 10);
if (!isNaN(value) && value >= 1 && value <= 100) {
setTaggerBatchSizeSaving(true);
void setTaggerBatchSize(value)
.catch((error: unknown) => setTaggerQueueStatus(String(error)))
.finally(() => {
setTaggerBatchSizeDraft(null);
setTaggerBatchSizeSaving(false);
});
} else {
setTaggerBatchSizeDraft(null);
}
}}
/>
<p className="text-[11px] text-gray-600">{taggerBatchSizeSaving ? "Saving..." : "Default: 8"}</p>
</div>
</SettingsRow>
<div>
<p className="text-xs font-medium text-gray-400">Model location</p>
<p className="mt-2 break-all rounded-md border border-white/[0.07] bg-black/20 px-3 py-2 text-xs text-gray-600">
{taggerReady ? taggerModelStatus?.local_dir : "Not downloaded"}
</p>
{taggerModelProgress?.current_file ? <p className="mt-3 break-all text-xs text-gray-500">{taggerModelProgress.current_file}</p> : null}
{taggerModelError ? <p className="mt-3 text-xs text-amber-300">{taggerModelError}</p> : null}
{taggerRuntimeProbe ? (
<div className="mt-4 rounded-lg border border-white/[0.07] bg-black/20 px-3 py-3">
<div className="flex items-center justify-between gap-3">
<p className="text-xs font-medium text-gray-400">Runtime check</p>
<StatusPill tone="ready">Ready</StatusPill>
</div>
<p className="mt-2 text-xs text-gray-600">Tagger acceleration: {taggerRuntimeProbe.acceleration}</p>
<p className="mt-2 break-all text-xs text-gray-500">{taggerRuntimeProbe.session.file}</p>
</div>
) : null}
</div>
</div>
</div>
</SettingsCard>
<SettingsCard title="Queue Targets" description="Choose which folders to include when queuing tagging jobs.">
<SettingsRow title="Target scope" description="Queue across the full library, or choose a specific folder set and keep the modal open while you work through it.">
<div className="flex rounded-lg border border-white/[0.07] bg-black/20 p-1">
<ScopeButton scope="all" current={taggingQueueScope} onSelect={setTaggingQueueScope}>All media</ScopeButton>
<ScopeButton scope="selected" current={taggingQueueScope} onSelect={setTaggingQueueScope}>Selected folders</ScopeButton>
</div>
</SettingsRow>
<div className="border-b border-white/[0.07] py-4">
<div className="flex items-center justify-between gap-4">
<div>
<p className="text-sm font-medium text-white">Folder selection</p>
<p className="mt-1 text-xs leading-relaxed text-gray-500">Current target: {queueScopeLabel}.</p>
</div>
<div className="flex gap-2">
<button
className="rounded-md border border-white/10 bg-white/[0.045] px-2.5 py-1 text-[11px] text-gray-400 transition-colors hover:bg-white/[0.075] hover:text-white disabled:opacity-40"
onClick={() => setTaggingQueueFolderIds(folders.map((folder) => folder.id))}
disabled={folders.length === 0}
>
Select all
</button>
<button
className="rounded-md border border-white/10 bg-white/[0.045] px-2.5 py-1 text-[11px] text-gray-400 transition-colors hover:bg-white/[0.075] hover:text-white disabled:opacity-40"
onClick={() => setTaggingQueueFolderIds([])}
disabled={taggingQueueFolderIds.length === 0}
>
Clear
</button>
</div>
</div>
<div className={`mt-4 grid max-h-64 gap-2 overflow-y-auto pr-1 ${taggingQueueScope === "selected" ? "opacity-100" : "opacity-60"}`}>
{folders.map((folder) => {
const active = taggingQueueFolderIds.includes(folder.id);
const progress = mediaJobProgress[folder.id];
return (
<button
key={folder.id}
type="button"
className={`flex items-center justify-between rounded-xl border px-3 py-2 text-left transition-colors ${
active
? "border-emerald-400/30 bg-emerald-500/10 text-white"
: "border-white/[0.07] bg-black/20 text-gray-300 hover:border-white/15 hover:bg-white/[0.04]"
}`}
onClick={() => toggleTaggingQueueFolder(folder.id)}
>
<div>
<p className="text-sm font-medium">{folder.name}</p>
<p className="mt-1 text-[11px] text-gray-500">{folder.image_count.toLocaleString()} items</p>
</div>
<div className="flex items-center gap-2">
{(progress?.tagging_pending ?? 0) > 0 ? <StatusPill tone="busy">{progress?.tagging_pending} queued</StatusPill> : null}
<span className={`h-4 w-4 rounded-full border ${active ? "border-emerald-300 bg-emerald-300" : "border-white/15 bg-transparent"}`} />
</div>
</button>
);
})}
{folders.length === 0 ? <p className="text-sm text-gray-500">Add a folder first to enable targeted tagging queues.</p> : null}
</div>
</div>
<SettingsRow title="Queue tagging jobs" description="Generate missing AI tags for the current target. Results flow back into the library as the background worker finishes.">
<div className="flex flex-col items-end gap-2">
<button
className="rounded-md border border-white/10 bg-white/[0.055] px-3 py-1.5 text-xs text-gray-300 transition-colors hover:bg-white/10 hover:text-white disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => runQueueAction("queue")}
disabled={!taggerReady || taggerQueueing || taggerClearing || (taggingQueueScope === "selected" && taggingQueueFolderIds.length === 0)}
>
{taggerQueueing ? "Queueing..." : "Queue tagging"}
</button>
<button
className="rounded-md border border-amber-400/20 bg-amber-500/10 px-3 py-1.5 text-xs text-amber-200 transition-colors hover:bg-amber-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => runQueueAction("clear")}
disabled={taggerQueueing || taggerClearing || (taggingQueueScope === "selected" && taggingQueueFolderIds.length === 0)}
>
{taggerClearing ? "Clearing..." : "Clear queued jobs"}
</button>
</div>
</SettingsRow>
{taggerQueueStatus ? <p className="pt-4 text-xs text-gray-500">{taggerQueueStatus}</p> : null}
</SettingsCard>
<SettingsCard title="Captioning">
<div className="rounded-xl border border-dashed border-white/[0.09] bg-black/20 px-4 py-4">
<p className="text-sm font-medium text-white/50">Coming soon</p>
</div>
</SettingsCard>
</SectionShell> </SectionShell>
); </div>
})() : null} ) : (
<div className="space-y-8">
<SectionShell
eyebrow="Workers"
title="Background processing"
>
<SettingsCard title="Queue summary" description="Live totals across all folder workers.">
<div className="grid gap-3 sm:grid-cols-3">
<div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
<p className="text-[11px] uppercase tracking-[0.14em] text-gray-600">Tagging queued</p>
<p className="mt-2 text-2xl font-semibold text-white">{totalQueuedJobs.toLocaleString()}</p>
</div>
<div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
<p className="text-[11px] uppercase tracking-[0.14em] text-gray-600">Selected folders</p>
<p className="mt-2 text-2xl font-semibold text-white">{selectedFolders.length.toLocaleString()}</p>
</div>
<div className="rounded-xl border border-white/[0.07] bg-black/20 p-4">
<p className="text-[11px] uppercase tracking-[0.14em] text-gray-600">Library folders</p>
<p className="mt-2 text-2xl font-semibold text-white">{folders.length.toLocaleString()}</p>
</div>
</div>
</SettingsCard>
{activeSection === "library" ? ( <SettingsCard title="Worker controls" description="Pause, resume, retry, and per-folder failure details are available in the background tasks panel.">
<SectionShell eyebrow="Library" title="Indexing and scanning"> <div className="flex items-center justify-between gap-4 rounded-xl border border-white/[0.07] bg-black/20 p-4">
<SettingsRow title="Background workers" description="Folder-level pause controls remain in the background tasks panel."> <p className="text-sm text-gray-400">Workers are running in the background.</p>
<StatusPill tone="muted">Managed per folder</StatusPill> <StatusPill tone="ready">Live</StatusPill>
</SettingsRow> </div>
<SettingsRow title="Reindexing" description="Use the library sidebar to rescan a folder when files change."> </SettingsCard>
<StatusPill tone="muted">Available</StatusPill> </SectionShell>
</SettingsRow> </div>
</SectionShell> )}
) : null}
{activeSection === "display" ? (
<SectionShell eyebrow="Display" title="Gallery preferences">
<SettingsRow title="Grid density" description="Use the toolbar size control to change thumbnail density.">
<StatusPill tone="muted">Toolbar</StatusPill>
</SettingsRow>
<SettingsRow title="Result view" description="Similar results reset to the top on each new search.">
<StatusPill tone="ready">Enabled</StatusPill>
</SettingsRow>
</SectionShell>
) : null}
{activeSection === "storage" ? (
<SectionShell eyebrow="Storage" title="Local files">
<SettingsRow title="WD Tagger" description="Remove the local tagger model without changing the rest of the library.">
<button
className="rounded-md border border-red-400/20 bg-red-500/10 px-3 py-1.5 text-xs text-red-200 transition-colors hover:bg-red-500/15 disabled:cursor-not-allowed disabled:opacity-45"
onClick={() => void deleteTaggerModel()}
disabled={taggerModelPreparing || !(taggerModelStatus?.ready ?? false)}
>
Delete model files
</button>
</SettingsRow>
</SectionShell>
) : null}
</div> </div>
</main> </main>
</div> </div>
+17
View File
@@ -138,6 +138,23 @@ export function Sidebar() {
Explore Explore
</span> </span>
</div> </div>
<div
className={`flex items-center gap-2.5 px-3 py-2 rounded-lg cursor-pointer transition-all duration-150 ${
activeView === "duplicates"
? "bg-white/8 text-white"
: "text-gray-500 hover:text-gray-200 hover:bg-white/5"
}`}
onClick={() => setView("duplicates")}
>
<svg className="w-3.5 h-3.5 shrink-0" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5}
d="M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z" />
</svg>
<span className={`text-[13px] font-medium ${activeView === "duplicates" ? "text-white" : ""}`}>
Duplicates
</span>
</div>
</div> </div>
{/* Section label */} {/* Section label */}
+322 -190
View File
@@ -1,243 +1,375 @@
import { useEffect } from "react"; import { useEffect, useLayoutEffect, useMemo, useRef, useState } from "react";
import { motion } from "framer-motion"; import { motion } from "framer-motion";
import { convertFileSrc } from "@tauri-apps/api/core"; import { convertFileSrc } from "@tauri-apps/api/core";
import { useGalleryStore, TagCloudEntry } from "../store"; import { ExploreTagEntry, TagCloudEntry, useGalleryStore } from "../store";
// Accent glow colours for the hover ring — cycled by index const ACCENTS = [
const GLOWS: string[] = [ "#60a5fa",
"rgba(59,130,246,0.5)", "#c084fc",
"rgba(168,85,247,0.5)", "#4ade80",
"rgba(16,185,129,0.5)", "#fbbf24",
"rgba(245,158,11,0.5)", "#f472b4",
"rgba(236,72,153,0.5)", "#2dd4bf",
"rgba(6,182,212,0.5)", "#fb923c",
"rgba(249,115,22,0.5)", "#a78bfa",
"rgba(34,197,94,0.5)", "#34d399",
"#f87171",
]; ];
function pseudoRandom(seed: number): number { const GOLDEN_ANGLE = Math.PI * (3 - Math.sqrt(5));
const x = Math.sin(seed + 1) * 10000;
function seeded(n: number): number {
const x = Math.sin(n * 9301 + 49297) * 233280;
return x - Math.floor(x); return x - Math.floor(x);
} }
// Map cluster size to a tile size bucket (px) interface PlacedNode {
function getTileSize(count: number, maxCount: number): number {
if (maxCount === 0) return 72;
const ratio = count / maxCount;
if (ratio > 0.75) return 160;
if (ratio > 0.45) return 128;
if (ratio > 0.22) return 104;
if (ratio > 0.08) return 88;
return 72;
}
function TagButton({
entry,
index,
maxCount,
onSearch,
}: {
entry: TagCloudEntry; entry: TagCloudEntry;
index: number; index: number;
maxCount: number; x: number;
onSearch: (imageId: number) => void; y: number;
}) { w: number;
const size = getTileSize(entry.count, maxCount); h: number;
const glow = GLOWS[index % GLOWS.length]; accent: string;
driftX: number;
driftY: number;
driftDuration: number;
rotateSeed: number;
}
// Small random rotation for organic feel — larger tiles stay flatter function buildCloud(entries: TagCloudEntry[], containerW: number, containerH: number): PlacedNode[] {
const maxRot = size >= 128 ? 0 : size >= 104 ? 3 : size >= 88 ? 6 : 10; if (!entries.length || containerW <= 0 || containerH <= 0) return [];
const rotation = (pseudoRandom(index * 7) - 0.5) * 2 * maxRot;
const mt = Math.floor(pseudoRandom(index * 3) * 10) + 4; const maxCount = Math.max(...entries.map((e) => e.count));
const mr = Math.floor(pseudoRandom(index * 5) * 12) + 4; const cx = containerW / 2;
const mb = Math.floor(pseudoRandom(index * 11) * 10) + 4; const cy = containerH / 2;
const ml = Math.floor(pseudoRandom(index * 13) * 12) + 4; // Spread ellipse shrinks slightly to leave room for card half-widths at the edges
const spreadX = containerW * 0.42;
const spreadY = containerH * 0.36;
const n = entries.length;
const src = entry.thumbnail_path ? convertFileSrc(entry.thumbnail_path) : null; // 1. Build initial positions using phyllotaxis spiral
const nodes: PlacedNode[] = entries.map((entry, i) => {
const ratio = Math.max(entry.count / maxCount, 0.08);
// Cards scale from 110px to 230px wide; height is 3/4 of width
const w = 110 + Math.sqrt(ratio) * 120;
const h = w * 0.75;
const radialRatio = Math.sqrt((i + 0.5) / n);
const angle = i * GOLDEN_ANGLE;
return {
entry,
index: i,
x: cx + Math.cos(angle) * radialRatio * spreadX,
y: cy + Math.sin(angle) * radialRatio * spreadY,
w,
h,
accent: ACCENTS[i % ACCENTS.length],
driftX: (seeded(i + 11) - 0.5) * 18,
driftY: (seeded(i + 17) - 0.5) * 14,
driftDuration: 8 + seeded(i + 23) * 7,
rotateSeed: (seeded(i + 31) - 0.5) * 4,
};
});
// 2. Iterative overlap resolution — no physics, just push apart
const PAD = 24;
for (let iter = 0; iter < 80; iter++) {
for (let a = 0; a < nodes.length; a++) {
const na = nodes[a];
for (let b = a + 1; b < nodes.length; b++) {
const nb = nodes[b];
const dx = nb.x - na.x;
const dy = nb.y - na.y;
const overlapX = (na.w + nb.w) / 2 + PAD - Math.abs(dx);
const overlapY = (na.h + nb.h) / 2 + PAD - Math.abs(dy);
if (overlapX <= 0 || overlapY <= 0) continue;
// Push along the smaller overlap axis
if (overlapX < overlapY) {
const push = overlapX * 0.5 * (dx >= 0 ? 1 : -1);
nb.x += push;
na.x -= push;
} else {
const push = overlapY * 0.5 * (dy >= 0 ? 1 : -1);
nb.y += push;
na.y -= push;
}
}
// Pull gently back toward anchor to prevent runaway drift
na.x += (cx + Math.cos(na.index * GOLDEN_ANGLE) * Math.sqrt((na.index + 0.5) / n) * spreadX - na.x) * 0.05;
na.y += (cy + Math.sin(na.index * GOLDEN_ANGLE) * Math.sqrt((na.index + 0.5) / n) * spreadY - na.y) * 0.05;
}
}
// 3. Clamp so cards never poke outside the container
return nodes.map((node) => ({
...node,
x: Math.min(Math.max(node.x, node.w / 2 + 16), containerW - node.w / 2 - 16),
y: Math.min(Math.max(node.y, node.h / 2 + 16), containerH - node.h / 2 - 16),
}));
}
function CloudCard({ node, onOpen }: { node: PlacedNode; onOpen: (imageIds: number[]) => void }) {
const src = node.entry.thumbnail_path ? convertFileSrc(node.entry.thumbnail_path) : null;
const { w, h, accent } = node;
return ( return (
<motion.button <motion.button
initial={{ opacity: 0, scale: 0.5, rotate: rotation * 2 }} className="group absolute overflow-hidden rounded-2xl border border-white/8 bg-white/[0.04] text-left shadow-[0_8px_40px_rgba(0,0,0,0.5)] focus:outline-none focus-visible:ring-2 focus-visible:ring-white/30"
animate={{ opacity: 1, scale: 1, rotate: rotation }} style={{ width: w, height: h, left: node.x - w / 2, top: node.y - h / 2 }}
initial={{ opacity: 0, scale: 0.75 }}
animate={{
opacity: 1,
scale: 1,
x: [0, node.driftX, 0],
y: [0, node.driftY, 0],
rotate: [node.rotateSeed, node.rotateSeed + 1.4, node.rotateSeed],
}}
transition={{ transition={{
delay: index * 0.025, opacity: { duration: 0.3, delay: Math.min(node.index * 0.035, 0.7) },
type: "spring", scale: { duration: 0.3, delay: Math.min(node.index * 0.035, 0.7) },
stiffness: 200, x: { duration: node.driftDuration, repeat: Infinity, ease: "easeInOut", delay: seeded(node.index + 41) * 3 },
damping: 18, y: { duration: node.driftDuration + 1.6, repeat: Infinity, ease: "easeInOut", delay: seeded(node.index + 51) * 3 },
}} rotate: { duration: node.driftDuration + 0.9, repeat: Infinity, ease: "easeInOut" },
whileHover={{
scale: 1.12,
rotate: 0,
transition: { type: "spring", stiffness: 400, damping: 22 },
}}
whileTap={{ scale: 0.92 }}
onClick={() => onSearch(entry.representative_image_id)}
title={`${entry.count} similar ${entry.count === 1 ? "photo" : "photos"}`}
style={{
width: size,
height: size,
margin: `${mt}px ${mr}px ${mb}px ${ml}px`,
borderRadius: 12,
border: "2px solid rgba(255,255,255,0.08)",
background: "rgba(255,255,255,0.04)",
cursor: "pointer",
padding: 0,
overflow: "hidden",
position: "relative",
flexShrink: 0,
boxShadow: "none",
transition: "border-color 0.15s, box-shadow 0.15s",
}}
onMouseEnter={(e) => {
const el = e.currentTarget;
el.style.borderColor = glow;
el.style.boxShadow = `0 0 16px ${glow}, 0 0 32px ${glow.replace("0.5", "0.25")}`;
}}
onMouseLeave={(e) => {
const el = e.currentTarget;
el.style.borderColor = "rgba(255,255,255,0.08)";
el.style.boxShadow = "none";
}} }}
whileHover={{ scale: 1.06, rotate: 0, transition: { duration: 0.18 } }}
onClick={() => onOpen(node.entry.image_ids)}
title={`Open cluster — ${node.entry.count.toLocaleString()} images`}
> >
{src ? ( {src ? (
<img <img
src={src} src={src}
alt="" alt=""
className="absolute inset-0 h-full w-full object-cover transition-transform duration-500 group-hover:scale-105"
draggable={false} draggable={false}
style={{
width: "100%",
height: "100%",
objectFit: "cover",
display: "block",
}}
/> />
) : ( ) : (
// Fallback placeholder when no thumbnail exists yet <div className="absolute inset-0 bg-gradient-to-br from-white/[0.07] to-transparent" />
<div
style={{
width: "100%",
height: "100%",
background: "rgba(255,255,255,0.06)",
}}
/>
)} )}
<div className="absolute inset-0 bg-gradient-to-t from-black/85 via-black/20 to-transparent" />
{/* Count badge — bottom-right corner */} {/* Accent glow on hover */}
<div <div
style={{ className="absolute inset-0 opacity-0 transition-opacity duration-300 group-hover:opacity-100"
position: "absolute", style={{ background: `radial-gradient(ellipse at bottom, ${accent}25, transparent 70%)` }}
bottom: 5, />
right: 5, <div className="absolute inset-x-0 bottom-0 p-3">
background: "rgba(0,0,0,0.6)", <div className="mb-2 h-px rounded-full" style={{ background: `linear-gradient(to right, ${accent}80, transparent)` }} />
color: "rgba(255,255,255,0.85)", <div className="flex items-end justify-between gap-2">
fontSize: 10, <div>
fontWeight: 600, <p className="text-[9px] uppercase tracking-[0.18em] text-white/35">Cluster</p>
lineHeight: 1, <p className="text-base font-semibold leading-none text-white">{node.entry.count.toLocaleString()}</p>
padding: "3px 6px", </div>
borderRadius: 6, <span
backdropFilter: "blur(4px)", className="rounded-full border px-2 py-0.5 text-[9px] uppercase tracking-[0.1em] opacity-0 transition-opacity duration-200 group-hover:opacity-100"
pointerEvents: "none", style={{ borderColor: `${accent}50`, color: accent, backgroundColor: `${accent}12` }}
}} >
> Open
{entry.count} </span>
</div>
</div> </div>
</motion.button> </motion.button>
); );
} }
// Actual tag cloud — word size driven by log-scaled frequency
function TagWord({
entry,
index,
logMin,
logRange,
onSearch,
}: {
entry: ExploreTagEntry;
index: number;
logMin: number;
logRange: number;
onSearch: (tag: string) => void;
}) {
const ratio = logRange > 0 ? (Math.log(Math.max(entry.count, 1)) - logMin) / logRange : 0.5;
const fontSize = 11 + ratio * 28; // 11px 39px
const accent = ACCENTS[index % ACCENTS.length];
const tilt = (seeded(index + 5) - 0.5) * 7;
return (
<motion.button
initial={{ opacity: 0, scale: 0.6 }}
animate={{ opacity: 0.4 + ratio * 0.6, scale: 1 }}
transition={{ delay: Math.min(index * 0.008, 0.55), duration: 0.22 }}
whileHover={{ scale: 1.2, opacity: 1, rotate: 0, transition: { duration: 0.14 } }}
className="group inline-flex items-center gap-1 rounded-full px-2 py-1 transition-colors hover:bg-white/[0.07]"
style={{ fontSize, rotate: tilt }}
onClick={() => onSearch(entry.tag)}
title={`${entry.tag}${entry.count.toLocaleString()} images`}
>
<span
className="font-medium leading-none"
style={{ color: ratio > 0.55 ? accent : "rgba(255,255,255,0.82)" }}
>
{entry.tag}
</span>
<span
className="rounded-full px-1.5 py-0.5 text-[9px] tabular-nums opacity-0 transition-opacity group-hover:opacity-100"
style={{ backgroundColor: `${accent}22`, color: accent }}
>
{entry.count.toLocaleString()}
</span>
</motion.button>
);
}
function Spinner() {
return (
<motion.div
className="h-5 w-5 rounded-full border-2 border-white/15 border-t-white/50"
animate={{ rotate: 360 }}
transition={{ duration: 0.85, repeat: Infinity, ease: "linear" }}
/>
);
}
// Separate component so its useLayoutEffect fires when the canvas is actually
// mounted — not at TagCloud mount time when the container may still be hidden
// behind a loading state.
function ClusterCloud({
entries,
onOpen,
}: {
entries: TagCloudEntry[];
onOpen: (imageIds: number[]) => void;
}) {
const canvasRef = useRef<HTMLDivElement>(null);
const [canvasSize, setCanvasSize] = useState({ w: 0, h: 0 });
useLayoutEffect(() => {
const el = canvasRef.current;
if (!el) return;
const update = () => {
const r = el.getBoundingClientRect();
setCanvasSize({ w: r.width, h: r.height });
};
update();
const ro = new ResizeObserver(update);
ro.observe(el);
return () => ro.disconnect();
}, []);
const nodes = useMemo(
() => buildCloud(entries, canvasSize.w, canvasSize.h),
[entries, canvasSize.w, canvasSize.h],
);
return (
<div ref={canvasRef} className="relative min-h-0 flex-1 overflow-hidden">
<div className="pointer-events-none absolute inset-0 opacity-[0.07] [background-image:radial-gradient(circle,rgba(255,255,255,0.6)_1px,transparent_1px)] [background-size:28px_28px]" />
{nodes.map((node) => (
<CloudCard key={`${node.entry.representative_image_id}:${node.index}`} node={node} onOpen={onOpen} />
))}
</div>
);
}
export function TagCloud() { export function TagCloud() {
const exploreMode = useGalleryStore((state) => state.exploreMode);
const setExploreMode = useGalleryStore((state) => state.setExploreMode);
const tagCloudEntries = useGalleryStore((state) => state.tagCloudEntries); const tagCloudEntries = useGalleryStore((state) => state.tagCloudEntries);
const tagCloudLoading = useGalleryStore((state) => state.tagCloudLoading); const tagCloudLoading = useGalleryStore((state) => state.tagCloudLoading);
const loadTagCloud = useGalleryStore((state) => state.loadTagCloud); const loadTagCloud = useGalleryStore((state) => state.loadTagCloud);
const searchByTag = useGalleryStore((state) => state.searchByTag); const exploreTagEntries = useGalleryStore((state) => state.exploreTagEntries);
const exploreTagLoading = useGalleryStore((state) => state.exploreTagLoading);
const loadExploreTags = useGalleryStore((state) => state.loadExploreTags);
const showVisualCluster = useGalleryStore((state) => state.showVisualCluster);
const searchForTag = useGalleryStore((state) => state.searchForTag);
const selectedFolderId = useGalleryStore((state) => state.selectedFolderId); const selectedFolderId = useGalleryStore((state) => state.selectedFolderId);
useEffect(() => { useEffect(() => {
void loadTagCloud(); if (exploreMode === "visual") void loadTagCloud();
}, [selectedFolderId]); else void loadExploreTags();
}, [exploreMode, selectedFolderId, loadTagCloud, loadExploreTags]);
const maxCount = const { logMin, logRange } = useMemo(() => {
tagCloudEntries.length > 0 if (!exploreTagEntries.length) return { logMin: 0, logRange: 1 };
? Math.max(...tagCloudEntries.map((e) => e.count)) const logs = exploreTagEntries.map((e) => Math.log(Math.max(e.count, 1)));
: 1; const lo = Math.min(...logs);
const hi = Math.max(...logs);
return { logMin: lo, logRange: hi - lo || 1 };
}, [exploreTagEntries]);
const loading = exploreMode === "visual" ? tagCloudLoading : exploreTagLoading;
const hasEntries = exploreMode === "visual" ? tagCloudEntries.length > 0 : exploreTagEntries.length > 0;
const entryCount = exploreMode === "visual" ? tagCloudEntries.length : exploreTagEntries.length;
return ( return (
<div className="flex-1 flex flex-col items-center min-h-0 overflow-y-auto"> <div className="flex min-h-0 flex-1 flex-col overflow-hidden bg-[radial-gradient(ellipse_at_top,rgba(59,130,246,0.08),transparent_50%),radial-gradient(ellipse_at_80%_75%,rgba(168,85,247,0.07),transparent_40%),#07080f]">
{/* Header */} {/* Header */}
<motion.div <div className="shrink-0 border-b border-white/[0.05] px-6 py-4">
className="text-center pt-14 pb-8 shrink-0" <div className="flex items-center justify-between gap-4">
initial={{ opacity: 0, y: -10 }} <div className="min-w-0">
animate={{ opacity: 1, y: 0 }} <h2 className="text-[15px] font-semibold text-white">Explore</h2>
transition={{ duration: 0.35 }} <p className="mt-0.5 truncate text-[11px] text-white/30">
> {loading
<h2 className="text-[22px] font-semibold text-white/70 tracking-tight mb-2"> ? exploreMode === "visual" ? "Computing visual clusters…" : "Loading tags…"
Explore your library : hasEntries
</h2> ? exploreMode === "visual"
<p className="text-[13px] text-white/25"> ? `${entryCount} cluster${entryCount !== 1 ? "s" : ""} — click any to open`
Visual clusters from your photos sized by how many match : `${entryCount} tag${entryCount !== 1 ? "s" : ""} — click any to search`
</p> : exploreMode === "visual"
</motion.div> ? "No clusters — images need embeddings first"
: "No tags — run the AI tagger or add tags manually"}
{/* Loading */} </p>
{tagCloudLoading && ( </div>
<div className="flex-1 flex flex-col items-center justify-center gap-4"> <div className="flex shrink-0 rounded-lg border border-white/8 bg-white/[0.03] p-0.5">
<motion.svg <button
className="w-8 h-8 text-white/20" className={`rounded-md px-3 py-1.5 text-xs transition-colors ${
viewBox="0 0 24 24" exploreMode === "visual" ? "bg-white/10 text-white" : "text-gray-500 hover:text-gray-300"
fill="none" }`}
animate={{ rotate: 360 }} onClick={() => setExploreMode("visual")}
transition={{ duration: 1, repeat: Infinity, ease: "linear" }} >
> Clusters
<circle </button>
cx="12" <button
cy="12" className={`rounded-md px-3 py-1.5 text-xs transition-colors ${
r="10" exploreMode === "tags" ? "bg-white/10 text-white" : "text-gray-500 hover:text-gray-300"
stroke="currentColor" }`}
strokeWidth="2" onClick={() => setExploreMode("tags")}
strokeOpacity="0.2" >
/> Tag Cloud
<path </button>
d="M4 12a8 8 0 018-8" </div>
stroke="currentColor"
strokeWidth="2"
strokeLinecap="round"
/>
</motion.svg>
<p className="text-[12px] text-white/20">Clustering your library</p>
</div> </div>
)} </div>
{/* Empty state */} {loading ? (
{!tagCloudLoading && tagCloudEntries.length === 0 && ( <div className="flex flex-1 items-center justify-center gap-3 text-white/25">
<div className="flex-1 flex items-center justify-center"> <Spinner />
<p className="text-[13px] text-white/25 text-center max-w-xs leading-relaxed"> <span className="text-sm">{exploreMode === "visual" ? "Computing clusters…" : "Loading tags…"}</span>
No embeddings yet. Add a folder and wait for the embedding worker to </div>
finish, then come back here. ) : !hasEntries ? (
<div className="flex flex-1 items-center justify-center px-8">
<p className="max-w-xs text-center text-sm leading-relaxed text-white/25">
{exploreMode === "visual"
? "No visual clusters yet. Images need embeddings before they can be grouped. Check indexing progress in the sidebar."
: "No tags yet. Run the AI tagger from Settings, or add tags manually in the image preview."}
</p> </p>
</div> </div>
)} ) : exploreMode === "visual" ? (
<ClusterCloud entries={tagCloudEntries} onOpen={showVisualCluster} />
{/* Cluster grid */} ) : (
{!tagCloudLoading && tagCloudEntries.length > 0 && ( /* Tag cloud — words sized by log-scaled frequency, wrapped freely */
<div className="flex flex-wrap justify-center px-12 pb-16 max-w-5xl w-full"> <div className="overflow-y-auto px-8 py-8">
{tagCloudEntries.map((entry, index) => ( <div className="flex flex-wrap items-center justify-center gap-x-0.5 gap-y-1 leading-none">
<TagButton {exploreTagEntries.map((entry, index) => (
key={entry.representative_image_id} <TagWord
entry={entry} key={entry.tag}
index={index} entry={entry}
maxCount={maxCount} index={index}
onSearch={searchByTag} logMin={logMin}
/> logRange={logRange}
))} onSearch={searchForTag}
/>
))}
</div>
</div> </div>
)} )}
{!tagCloudLoading && tagCloudEntries.length > 0 && (
<p className="shrink-0 pb-8 text-[11px] text-white/12 text-center">
Grouped by visual similarity · CLIP ViT-B/32
</p>
)}
</div> </div>
); );
} }
+213 -75
View File
@@ -1,11 +1,14 @@
import { useEffect, useRef, useState } from "react"; import { useEffect, useRef, useState } from "react";
import { tileSizeForZoom, useGalleryStore, SortOrder, MediaFilter, SearchMode } from "../store"; import { invoke } from "@tauri-apps/api/core";
import { tileSizeForZoom, useGalleryStore, SortOrder, MediaFilter, SearchCommand, parseSearchValue, searchModeLabel, ExploreTagEntry } from "../store";
const BASE_SORT_OPTIONS: { value: SortOrder; label: string }[] = [ const BASE_SORT_OPTIONS: { value: SortOrder; label: string }[] = [
{ value: "date_desc", label: "Newest first" }, { value: "date_desc", label: "Newest first" },
{ value: "date_asc", label: "Oldest first" }, { value: "date_asc", label: "Oldest first" },
{ value: "name_asc", label: "Name AZ" }, { value: "name_asc", label: "Name AZ" },
{ value: "name_desc", label: "Name ZA" }, { value: "name_desc", label: "Name ZA" },
{ value: "rating_desc", label: "Highest rated" },
{ value: "rating_asc", label: "Lowest rated" },
{ value: "size_desc", label: "Largest first" }, { value: "size_desc", label: "Largest first" },
{ value: "size_asc", label: "Smallest first" }, { value: "size_asc", label: "Smallest first" },
]; ];
@@ -116,12 +119,27 @@ function FilterPill({
); );
} }
function commandPrefix(command: SearchCommand | null): string | null {
switch (command) {
case "semantic":
return "/s";
case "tag":
return "/t";
default:
return null;
}
}
function composeSearchValue(command: SearchCommand | null, query: string): string {
const prefix = commandPrefix(command);
if (!prefix) return query;
return query.length > 0 ? `${prefix} ${query}` : prefix;
}
export function Toolbar() { export function Toolbar() {
const search = useGalleryStore((state) => state.search); const search = useGalleryStore((state) => state.search);
const setSearch = useGalleryStore((state) => state.setSearch); const setSearch = useGalleryStore((state) => state.setSearch);
const clearSearch = useGalleryStore((state) => state.clearSearch); const clearSearch = useGalleryStore((state) => state.clearSearch);
const searchMode = useGalleryStore((state) => state.searchMode);
const setSearchMode = useGalleryStore((state) => state.setSearchMode);
const sort = useGalleryStore((state) => state.sort); const sort = useGalleryStore((state) => state.sort);
const setSort = useGalleryStore((state) => state.setSort); const setSort = useGalleryStore((state) => state.setSort);
const totalImages = useGalleryStore((state) => state.totalImages); const totalImages = useGalleryStore((state) => state.totalImages);
@@ -133,20 +151,26 @@ export function Toolbar() {
const setMediaFilter = useGalleryStore((state) => state.setMediaFilter); const setMediaFilter = useGalleryStore((state) => state.setMediaFilter);
const favoritesOnly = useGalleryStore((state) => state.favoritesOnly); const favoritesOnly = useGalleryStore((state) => state.favoritesOnly);
const setFavoritesOnly = useGalleryStore((state) => state.setFavoritesOnly); const setFavoritesOnly = useGalleryStore((state) => state.setFavoritesOnly);
const minimumRating = useGalleryStore((state) => state.minimumRating);
const setMinimumRating = useGalleryStore((state) => state.setMinimumRating);
const failedEmbeddingsOnly = useGalleryStore((state) => state.failedEmbeddingsOnly); const failedEmbeddingsOnly = useGalleryStore((state) => state.failedEmbeddingsOnly);
const setFailedEmbeddingsOnly = useGalleryStore((state) => state.setFailedEmbeddingsOnly); const setFailedEmbeddingsOnly = useGalleryStore((state) => state.setFailedEmbeddingsOnly);
const similarScope = useGalleryStore((state) => state.similarScope);
const setSimilarScope = useGalleryStore((state) => state.setSimilarScope);
const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress); const mediaJobProgress = useGalleryStore((state) => state.mediaJobProgress);
const zoomPreset = useGalleryStore((state) => state.zoomPreset); const zoomPreset = useGalleryStore((state) => state.zoomPreset);
const setZoomPreset = useGalleryStore((state) => state.setZoomPreset); const setZoomPreset = useGalleryStore((state) => state.setZoomPreset);
const hasAnyFailedEmbeddings = Object.values(mediaJobProgress).some((p) => p.embedding_failed > 0); const hasAnyFailedEmbeddings = Object.values(mediaJobProgress).some((p) => p.embedding_failed > 0);
const [searchValue, setSearchValue] = useState(search); const [searchCommand, setSearchCommand] = useState<SearchCommand | null>(null);
const [searchQuery, setSearchQuery] = useState(search);
const [searchPanelOpen, setSearchPanelOpen] = useState(false);
const [tagSuggestions, setTagSuggestions] = useState<ExploreTagEntry[]>([]);
const debounceRef = useRef<ReturnType<typeof setTimeout> | null>(null); const debounceRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const suggestDebounceRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const searchInputRef = useRef<HTMLInputElement>(null); const searchInputRef = useRef<HTMLInputElement>(null);
// Tracks whether the user has typed in the search box at least once. const searchShellRef = useRef<HTMLDivElement>(null);
// Prevents the debounce effect from dispatching setSearch on initial mount
// when searchValue === search (which would wipe a loadSimilarImages result).
const userHasTyped = useRef(false); const userHasTyped = useRef(false);
const selectedFolder = folders.find((folder) => folder.id === selectedFolderId); const selectedFolder = folders.find((folder) => folder.id === selectedFolderId);
@@ -154,11 +178,8 @@ export function Toolbar() {
const tileSize = tileSizeForZoom(zoomPreset); const tileSize = tileSizeForZoom(zoomPreset);
const sortOptions = getSortOptions(mediaFilter); const sortOptions = getSortOptions(mediaFilter);
const hasActiveSearch = search.trim().length > 0; const hasActiveSearch = search.trim().length > 0;
const parsedSearch = parseSearchValue(composeSearchValue(searchCommand, searchQuery));
const searchModes: { value: SearchMode; label: string }[] = [ const isSimilarResults = collectionTitle === "Similar Images";
{ value: "filename", label: "Filename" },
{ value: "semantic", label: "Semantic" },
];
// If current sort is video-only but we switched away from video filter, reset to date_desc // If current sort is video-only but we switched away from video filter, reset to date_desc
useEffect(() => { useEffect(() => {
@@ -170,35 +191,62 @@ export function Toolbar() {
useEffect(() => { useEffect(() => {
if (!userHasTyped.current) return; if (!userHasTyped.current) return;
if (debounceRef.current) clearTimeout(debounceRef.current); if (debounceRef.current) clearTimeout(debounceRef.current);
debounceRef.current = setTimeout(() => { setSearch(searchValue); }, 200); debounceRef.current = setTimeout(() => { setSearch(composeSearchValue(searchCommand, searchQuery)); }, 200);
return () => { if (debounceRef.current) clearTimeout(debounceRef.current); }; return () => { if (debounceRef.current) clearTimeout(debounceRef.current); };
}, [searchValue, setSearch]); }, [searchCommand, searchQuery, setSearch]);
useEffect(() => { useEffect(() => {
setSearchValue(search); const parsed = parseSearchValue(search);
setSearchCommand(parsed.prefix && parsed.mode !== "filename" ? parsed.mode : null);
setSearchQuery(parsed.prefix ? parsed.query : search);
}, [search]); }, [search]);
// Fetch tag suggestions when in tag mode
useEffect(() => {
if (searchCommand !== "tag") {
setTagSuggestions([]);
return;
}
if (suggestDebounceRef.current) clearTimeout(suggestDebounceRef.current);
suggestDebounceRef.current = setTimeout(async () => {
try {
const results = await invoke<ExploreTagEntry[]>("search_tags_autocomplete", {
params: { query: searchQuery.trim(), folder_id: selectedFolderId ?? null, limit: 10 },
});
setTagSuggestions(results);
} catch {
setTagSuggestions([]);
}
}, 120);
return () => { if (suggestDebounceRef.current) clearTimeout(suggestDebounceRef.current); };
}, [searchCommand, searchQuery, selectedFolderId]);
useEffect(() => { useEffect(() => {
const handleKeyDown = (event: KeyboardEvent) => { const handleKeyDown = (event: KeyboardEvent) => {
const isModeToggle = (event.ctrlKey || event.metaKey) && event.shiftKey && event.key.toLowerCase() === "s";
if (isModeToggle) {
event.preventDefault();
setSearchMode(searchMode === "semantic" ? "filename" : "semantic");
searchInputRef.current?.focus();
return;
}
const activeElement = document.activeElement; const activeElement = document.activeElement;
const searchFocused = activeElement === searchInputRef.current; const searchFocused = activeElement === searchInputRef.current;
if (event.key === "Escape" && (searchFocused || hasActiveSearch)) { if (event.key === "Escape" && (searchFocused || hasActiveSearch)) {
event.preventDefault(); event.preventDefault();
setSearchValue(""); setSearchCommand(null);
setSearchQuery("");
clearSearch(); clearSearch();
} }
}; };
window.addEventListener("keydown", handleKeyDown); window.addEventListener("keydown", handleKeyDown);
return () => window.removeEventListener("keydown", handleKeyDown); return () => window.removeEventListener("keydown", handleKeyDown);
}, [clearSearch, hasActiveSearch, searchMode, setSearchMode]); }, [clearSearch, hasActiveSearch]);
useEffect(() => {
const close = (event: PointerEvent) => {
if (searchShellRef.current?.contains(event.target as Node)) return;
setSearchPanelOpen(false);
};
window.addEventListener("pointerdown", close);
return () => window.removeEventListener("pointerdown", close);
}, []);
const showTagSuggestions = searchCommand === "tag" && searchPanelOpen;
const showCommandHints = !searchCommand && searchPanelOpen;
return ( return (
<div className="relative z-20 shrink-0 border-b border-white/[0.06] bg-gray-950/80 backdrop-blur-xl"> <div className="relative z-20 shrink-0 border-b border-white/[0.06] bg-gray-950/80 backdrop-blur-xl">
@@ -212,9 +260,9 @@ export function Toolbar() {
? `${loadedCount.toLocaleString()} / ${totalImages.toLocaleString()}` ? `${loadedCount.toLocaleString()} / ${totalImages.toLocaleString()}`
: totalImages.toLocaleString()} : totalImages.toLocaleString()}
</span> </span>
{(hasActiveSearch || searchMode === "semantic") && ( {hasActiveSearch && (
<span className="rounded-md border border-blue-500/20 bg-blue-500/10 px-2 py-0.5 text-[10px] uppercase tracking-[0.14em] text-blue-300 shrink-0"> <span className="rounded-md border border-blue-500/20 bg-blue-500/10 px-2 py-0.5 text-[10px] uppercase tracking-[0.14em] text-blue-300 shrink-0">
{searchMode === "semantic" ? "Semantic Search" : "Filename Search"} {searchModeLabel(parsedSearch.mode)}
</span> </span>
)} )}
</div> </div>
@@ -222,55 +270,140 @@ export function Toolbar() {
<div className="flex-1" /> <div className="flex-1" />
{/* Search */} {/* Search */}
<div className="flex items-center rounded-lg border border-white/8 bg-white/5 overflow-hidden"> <div ref={searchShellRef} className="relative">
<div className="flex items-center pl-2 pr-1 gap-1 border-r border-white/8"> <div className="flex items-center rounded-lg border border-white/8 bg-white/5 overflow-hidden">
{searchModes.map((mode) => ( <div className="relative">
<button <svg className="pointer-events-none absolute left-2.5 top-1/2 h-3.5 w-3.5 -translate-y-1/2 text-gray-600"
key={mode.value} fill="none" viewBox="0 0 24 24" stroke="currentColor">
className={`rounded-md px-2 py-1 text-[11px] transition-colors ${ <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2}
searchMode === mode.value d="M21 21l-4.35-4.35M17 11A6 6 0 115 11a6 6 0 0112 0z" />
? "bg-white/10 text-white" </svg>
: "text-gray-500 hover:text-gray-200" <input
}`} ref={searchInputRef}
title={mode.value === "semantic" ? "Toggle with Ctrl/Cmd+Shift+S" : "Toggle with Ctrl/Cmd+Shift+S"} type="text"
onClick={() => setSearchMode(mode.value)} value={searchQuery}
> onChange={(event) => {
{mode.label} userHasTyped.current = true;
</button> const nextValue = event.target.value;
))} if (!searchCommand) {
</div> const parsed = parseSearchValue(nextValue);
<div className="relative"> if (parsed.prefix) {
<svg className="pointer-events-none absolute left-2.5 top-1/2 h-3.5 w-3.5 -translate-y-1/2 text-gray-600" setSearchCommand(parsed.mode === "filename" ? null : parsed.mode);
fill="none" viewBox="0 0 24 24" stroke="currentColor"> setSearchQuery(parsed.query);
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} return;
d="M21 21l-4.35-4.35M17 11A6 6 0 115 11a6 6 0 0112 0z" /> }
</svg> }
<input setSearchQuery(nextValue);
ref={searchInputRef}
type="text"
value={searchValue}
onChange={(event) => {
userHasTyped.current = true;
setSearchValue(event.target.value);
}}
placeholder={searchMode === "semantic" ? "Search by meaning..." : "Search filenames..."}
className="w-64 bg-transparent py-1.5 pl-8 pr-9 text-sm text-white placeholder:text-gray-600 focus:outline-none transition-colors"
/>
{searchValue.trim().length > 0 && (
<button
className="absolute right-2 top-1/2 -translate-y-1/2 rounded-md p-1 text-gray-500 hover:bg-white/5 hover:text-gray-200 transition-colors"
title="Clear search"
onClick={() => {
setSearchValue("");
clearSearch();
}} }}
> onKeyDown={(event) => {
<svg className="h-3.5 w-3.5" fill="none" viewBox="0 0 24 24" stroke="currentColor"> if (event.key === "Backspace" && searchQuery.length === 0 && searchCommand !== null) {
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" /> event.preventDefault();
</svg> setSearchCommand(null);
</button> }
)} }}
onFocus={() => setSearchPanelOpen(true)}
placeholder="Search files, or use /s /t"
className={`w-64 bg-transparent py-1.5 pr-9 text-sm text-white placeholder:text-gray-600 focus:outline-none transition-colors ${searchCommand !== null ? "pl-16" : "pl-8"}`}
/>
{searchCommand !== null ? (
<div className="absolute left-8 top-1/2 -translate-y-1/2">
<button
type="button"
className="rounded-md border border-white/10 bg-white/[0.05] px-1.5 py-0.5 font-mono text-[11px] text-gray-300 transition-colors hover:bg-white/[0.08] hover:text-white"
onClick={() => { setSearchCommand(null); setTagSuggestions([]); }}
title="Remove search command"
>
{commandPrefix(searchCommand)}
</button>
</div>
) : null}
{searchQuery.trim().length > 0 || searchCommand !== null ? (
<button
className="absolute right-2 top-1/2 -translate-y-1/2 rounded-md p-1 text-gray-500 hover:bg-white/5 hover:text-gray-200 transition-colors"
title="Clear search"
onClick={() => {
setSearchCommand(null);
setSearchQuery("");
setTagSuggestions([]);
clearSearch();
}}
>
<svg className="h-3.5 w-3.5" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
) : null}
</div>
</div> </div>
{/* Tag autocomplete suggestions */}
{showTagSuggestions && tagSuggestions.length > 0 ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-72 rounded-xl border border-white/10 bg-gray-950/98 py-1 shadow-2xl backdrop-blur">
{tagSuggestions.map((entry) => (
<button
key={entry.tag}
className="flex w-full items-center justify-between gap-3 px-3 py-2 text-left transition-colors hover:bg-white/[0.05]"
onMouseDown={(e) => {
// mousedown fires before input blur, so we prevent losing focus
e.preventDefault();
userHasTyped.current = true;
setSearchQuery(entry.tag);
setSearch(`/t ${entry.tag}`);
setSearchPanelOpen(false);
searchInputRef.current?.blur();
}}
>
<span className="text-sm text-white/88">{entry.tag}</span>
<span className="shrink-0 text-[11px] tabular-nums text-white/30">{entry.count.toLocaleString()}</span>
</button>
))}
</div>
) : null}
{/* Tag mode with no suggestions yet — show a brief hint */}
{showTagSuggestions && tagSuggestions.length === 0 && searchQuery.trim().length > 0 ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-72 rounded-xl border border-white/10 bg-gray-950/98 px-3 py-2.5 shadow-2xl backdrop-blur">
<p className="text-xs text-white/25">No matching tags</p>
</div>
) : null}
{/* Semantic mode hint */}
{searchCommand === "semantic" && searchPanelOpen ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-72 rounded-xl border border-white/10 bg-gray-950/98 px-3 py-2.5 shadow-2xl backdrop-blur">
<p className="text-xs text-white/40">Search by meaning and visual concepts</p>
</div>
) : null}
{/* Command hints — only shown when no command is active */}
{showCommandHints ? (
<div className="absolute left-0 top-full z-30 mt-1.5 w-64 rounded-xl border border-white/10 bg-gray-950/98 py-1 shadow-2xl backdrop-blur">
{(
[
{ command: "tag" as SearchCommand, prefix: "/t", label: "Tags", description: "Search AI and user tags" },
{ command: "semantic" as SearchCommand, prefix: "/s", label: "Semantic", description: "Search by meaning" },
] as const
).map((option) => (
<button
key={option.prefix}
className="flex w-full items-center gap-3 px-3 py-2 text-left transition-colors hover:bg-white/[0.05]"
onMouseDown={(e) => {
e.preventDefault();
userHasTyped.current = true;
setSearchCommand(option.command);
searchInputRef.current?.focus();
}}
>
<span className="rounded border border-white/10 bg-white/[0.04] px-1.5 py-0.5 font-mono text-[11px] text-gray-400">
{option.prefix}
</span>
<div>
<p className="text-sm text-gray-200">{option.label}</p>
<p className="text-xs text-gray-500">{option.description}</p>
</div>
</button>
))}
</div>
) : null}
</div> </div>
{/* Sort */} {/* Sort */}
@@ -302,10 +435,14 @@ export function Toolbar() {
{/* Filter row */} {/* Filter row */}
<div className="flex items-center gap-1 px-4 pb-1.5"> <div className="flex items-center gap-1 px-4 pb-1.5">
<FilterPill label="All" active={mediaFilter === "all" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("all"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} /> <FilterPill label="All" active={mediaFilter === "all" && !favoritesOnly && !failedEmbeddingsOnly && minimumRating === 0} onClick={() => { setMediaFilter("all"); setFavoritesOnly(false); setMinimumRating(0); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Images" active={mediaFilter === "image" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("image"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} /> <FilterPill label="Images" active={mediaFilter === "image" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("image"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Videos" active={mediaFilter === "video" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("video"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} /> <FilterPill label="Videos" active={mediaFilter === "video" && !favoritesOnly && !failedEmbeddingsOnly} onClick={() => { setMediaFilter("video"); setFavoritesOnly(false); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Favorites" active={favoritesOnly} onClick={() => { setFavoritesOnly(!favoritesOnly); setFailedEmbeddingsOnly(false); }} /> <FilterPill label="Favorites" active={favoritesOnly} onClick={() => { setFavoritesOnly(!favoritesOnly); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Rated" active={minimumRating === 1} onClick={() => { setMinimumRating(minimumRating === 1 ? 0 : 1); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="4★+" active={minimumRating === 4} onClick={() => { setMinimumRating(minimumRating === 4 ? 0 : 4); setFailedEmbeddingsOnly(false); }} />
<FilterPill label="Similar: Folder" active={similarScope === "current_folder"} onClick={() => setSimilarScope("current_folder")} />
<FilterPill label="Similar: All" active={similarScope === "all_media"} onClick={() => setSimilarScope("all_media")} />
{hasAnyFailedEmbeddings ? ( {hasAnyFailedEmbeddings ? (
<FilterPill <FilterPill
label="Failed Embeddings" label="Failed Embeddings"
@@ -314,6 +451,7 @@ export function Toolbar() {
onClick={() => setFailedEmbeddingsOnly(!failedEmbeddingsOnly)} onClick={() => setFailedEmbeddingsOnly(!failedEmbeddingsOnly)}
/> />
) : null} ) : null}
{isSimilarResults ? <span className="ml-2 text-[11px] text-gray-500">Current similar scope: {similarScope === "current_folder" ? "current folder" : "all media"}</span> : null}
</div> </div>
</div> </div>
); );
+546 -38
View File
@@ -15,10 +15,14 @@ export type MediaKind = "image" | "video";
export type MediaFilter = "all" | MediaKind; export type MediaFilter = "all" | MediaKind;
export type ZoomPreset = "compact" | "comfortable" | "detail"; export type ZoomPreset = "compact" | "comfortable" | "detail";
export type SearchMode = "filename" | "semantic"; export type SearchMode = "filename" | "semantic";
export type SearchCommand = "filename" | "semantic" | "tag";
export type CaptionAcceleration = "auto" | "cpu" | "directml"; export type CaptionAcceleration = "auto" | "cpu" | "directml";
export type CaptionDetail = "short" | "detailed" | "paragraph"; export type CaptionDetail = "short" | "detailed" | "paragraph";
export type TaggerAcceleration = "auto" | "cpu" | "directml"; export type TaggerAcceleration = "auto" | "cpu" | "directml";
export type AiRating = "general" | "sensitive" | "questionable" | "explicit"; export type AiRating = "general" | "sensitive" | "questionable" | "explicit";
export type TaggingQueueScope = "all" | "selected";
export type SimilarScope = "all_media" | "current_folder";
export type ExploreMode = "visual" | "tags";
export interface ImageRecord { export interface ImageRecord {
id: number; id: number;
@@ -115,12 +119,33 @@ export interface ThumbnailBatch {
images: ImageRecord[]; images: ImageRecord[];
} }
export type ActiveView = "gallery" | "explore"; export type ActiveView = "gallery" | "explore" | "duplicates";
export interface TagCloudEntry { export interface TagCloudEntry {
count: number; count: number;
representative_image_id: number; representative_image_id: number;
thumbnail_path: string | null; thumbnail_path: string | null;
image_ids: number[];
}
export interface ExploreTagEntry {
tag: string;
count: number;
representative_image_id: number;
thumbnail_path: string | null;
}
export interface DuplicateGroup {
file_hash: string;
file_size: number;
images: ImageRecord[];
}
export interface SimilarImagesPage {
images: ImageRecord[];
offset: number;
limit: number;
has_more: boolean;
} }
export interface CaptionModelStatus { export interface CaptionModelStatus {
@@ -171,6 +196,12 @@ export interface TaggerRuntimeProbe {
session: TaggerRuntimeSessionProbe; session: TaggerRuntimeSessionProbe;
} }
export interface ParsedSearch {
mode: SearchCommand;
query: string;
prefix: string | null;
}
export type SortOrder = export type SortOrder =
| "date_desc" | "date_desc"
| "date_asc" | "date_asc"
@@ -178,6 +209,8 @@ export type SortOrder =
| "name_desc" | "name_desc"
| "size_desc" | "size_desc"
| "size_asc" | "size_asc"
| "rating_desc"
| "rating_asc"
| "duration_desc" | "duration_desc"
| "duration_asc"; | "duration_asc";
@@ -194,17 +227,25 @@ interface GalleryState {
sort: SortOrder; sort: SortOrder;
mediaFilter: MediaFilter; mediaFilter: MediaFilter;
favoritesOnly: boolean; favoritesOnly: boolean;
minimumRating: number;
failedEmbeddingsOnly: boolean; failedEmbeddingsOnly: boolean;
zoomPreset: ZoomPreset; zoomPreset: ZoomPreset;
selectedImage: ImageRecord | null; selectedImage: ImageRecord | null;
collectionTitle: string | null; collectionTitle: string | null;
similarSourceImageId: number | null; similarSourceImageId: number | null;
similarSourceFolderId: number | null;
similarHasMore: boolean; similarHasMore: boolean;
similarScope: SimilarScope;
similarFolderId: number | null;
galleryScrollResetKey: number; galleryScrollResetKey: number;
activeView: ActiveView; activeView: ActiveView;
exploreMode: ExploreMode;
tagCloudEntries: TagCloudEntry[]; tagCloudEntries: TagCloudEntry[];
tagCloudLoading: boolean; tagCloudLoading: boolean;
tagCloudFolderId: number | null | undefined; // undefined = never loaded tagCloudFolderId: number | null | undefined; // undefined = never loaded
exploreTagEntries: ExploreTagEntry[];
exploreTagLoading: boolean;
exploreTagsFolderId: number | null | undefined;
indexingProgress: Record<number, IndexProgress>; indexingProgress: Record<number, IndexProgress>;
mediaJobProgress: Record<number, FolderJobProgress>; mediaJobProgress: Record<number, FolderJobProgress>;
cacheDir: string; cacheDir: string;
@@ -218,6 +259,8 @@ interface GalleryState {
captionDetail: CaptionDetail; captionDetail: CaptionDetail;
aiCaptionsEnabled: boolean; aiCaptionsEnabled: boolean;
settingsOpen: boolean; settingsOpen: boolean;
taggingQueueScope: TaggingQueueScope;
taggingQueueFolderIds: number[];
taggerModelStatus: TaggerModelStatus | null; taggerModelStatus: TaggerModelStatus | null;
taggerModelPreparing: boolean; taggerModelPreparing: boolean;
@@ -225,9 +268,16 @@ interface GalleryState {
taggerModelProgress: TaggerModelProgress | null; taggerModelProgress: TaggerModelProgress | null;
taggerAcceleration: TaggerAcceleration; taggerAcceleration: TaggerAcceleration;
taggerThreshold: number; taggerThreshold: number;
taggerBatchSize: number;
taggerRuntimeProbe: TaggerRuntimeProbe | null; taggerRuntimeProbe: TaggerRuntimeProbe | null;
taggerRuntimeChecking: boolean; taggerRuntimeChecking: boolean;
duplicateGroups: DuplicateGroup[];
duplicateScanning: boolean;
duplicateScanProgress: { scanned: number; total: number } | null;
duplicateSelectedIds: Set<number>;
duplicateLastScanned: number | null; // Unix timestamp (seconds)
loadFolders: () => Promise<void>; loadFolders: () => Promise<void>;
loadBackgroundJobProgress: () => Promise<void>; loadBackgroundJobProgress: () => Promise<void>;
addFolder: (path: string) => Promise<void>; addFolder: (path: string) => Promise<void>;
@@ -243,14 +293,20 @@ interface GalleryState {
setSort: (sort: SortOrder) => void; setSort: (sort: SortOrder) => void;
setMediaFilter: (filter: MediaFilter) => void; setMediaFilter: (filter: MediaFilter) => void;
setFavoritesOnly: (favoritesOnly: boolean) => void; setFavoritesOnly: (favoritesOnly: boolean) => void;
setMinimumRating: (minimumRating: number) => void;
setFailedEmbeddingsOnly: (failedEmbeddingsOnly: boolean) => void; setFailedEmbeddingsOnly: (failedEmbeddingsOnly: boolean) => void;
setZoomPreset: (zoomPreset: ZoomPreset) => void; setZoomPreset: (zoomPreset: ZoomPreset) => void;
openImage: (image: ImageRecord) => void; openImage: (image: ImageRecord) => void;
closeImage: () => void; closeImage: () => void;
setView: (view: ActiveView) => void; setView: (view: ActiveView) => void;
setExploreMode: (mode: ExploreMode) => void;
loadTagCloud: () => Promise<void>; loadTagCloud: () => Promise<void>;
searchByTag: (imageId: number) => void; loadExploreTags: () => Promise<void>;
loadSimilarImages: (imageId: number, folderId?: number | null, reset?: boolean) => Promise<void>; showVisualCluster: (imageIds: number[]) => Promise<void>;
searchForTag: (tag: string) => void;
loadSimilarImages: (imageId: number, folderId?: number | null, reset?: boolean, sourceFolderId?: number | null) => Promise<void>;
loadSimilarByRegion: (imageId: number, crop: { x: number; y: number; w: number; h: number }, folderId?: number | null, sourceFolderId?: number | null) => Promise<void>;
setSimilarScope: (scope: SimilarScope) => void;
suggestImageTags: (imageId: number) => Promise<string[]>; suggestImageTags: (imageId: number) => Promise<string[]>;
loadCaptionModelStatus: () => Promise<void>; loadCaptionModelStatus: () => Promise<void>;
prepareCaptionModel: () => Promise<void>; prepareCaptionModel: () => Promise<void>;
@@ -268,6 +324,9 @@ interface GalleryState {
setCaptionDetail: (detail: CaptionDetail) => Promise<void>; setCaptionDetail: (detail: CaptionDetail) => Promise<void>;
setAiCaptionsEnabled: (enabled: boolean) => void; setAiCaptionsEnabled: (enabled: boolean) => void;
setSettingsOpen: (open: boolean) => void; setSettingsOpen: (open: boolean) => void;
setTaggingQueueScope: (scope: TaggingQueueScope) => void;
toggleTaggingQueueFolder: (folderId: number) => void;
setTaggingQueueFolderIds: (folderIds: number[]) => void;
retryFailedEmbeddings: (folderId: number) => Promise<void>; retryFailedEmbeddings: (folderId: number) => Promise<void>;
updateImageDetails: (imageId: number, updates: { favorite?: boolean; rating?: number }) => Promise<void>; updateImageDetails: (imageId: number, updates: { favorite?: boolean; rating?: number }) => Promise<void>;
setCacheDir: (dir: string) => void; setCacheDir: (dir: string) => void;
@@ -280,10 +339,21 @@ interface GalleryState {
setTaggerAcceleration: (acceleration: TaggerAcceleration) => Promise<void>; setTaggerAcceleration: (acceleration: TaggerAcceleration) => Promise<void>;
loadTaggerThreshold: () => Promise<void>; loadTaggerThreshold: () => Promise<void>;
setTaggerThreshold: (threshold: number) => Promise<void>; setTaggerThreshold: (threshold: number) => Promise<void>;
loadTaggerBatchSize: () => Promise<void>;
setTaggerBatchSize: (batchSize: number) => Promise<void>;
probeTaggerRuntime: () => Promise<void>; probeTaggerRuntime: () => Promise<void>;
queueTaggingJobs: (folderId?: number | null) => Promise<number>; queueTaggingJobs: (folderId?: number | null) => Promise<number>;
queueTaggingJobsForFolders: (folderIds: number[]) => Promise<number>;
queueTaggingForImage: (imageId: number) => Promise<number>; queueTaggingForImage: (imageId: number) => Promise<number>;
clearTaggingJobs: (folderId?: number | null) => Promise<number>; clearTaggingJobs: (folderId?: number | null) => Promise<number>;
clearTaggingJobsForFolders: (folderIds: number[]) => Promise<number>;
loadDuplicateScanCache: (folderId?: number | null) => Promise<void>;
scanDuplicates: (folderId?: number | null) => Promise<void>;
toggleDuplicateSelected: (imageId: number) => void;
selectAllDuplicates: (imageIds: number[]) => void;
selectKeepFirstAllGroups: () => void;
clearDuplicateSelection: () => void;
deleteSelectedDuplicates: () => Promise<number>;
getImageTags: (imageId: number) => Promise<ImageTag[]>; getImageTags: (imageId: number) => Promise<ImageTag[]>;
addUserTag: (imageId: number, tag: string) => Promise<ImageTag>; addUserTag: (imageId: number, tag: string) => Promise<ImageTag>;
removeTag: (tagId: number) => Promise<void>; removeTag: (tagId: number) => Promise<void>;
@@ -291,6 +361,12 @@ interface GalleryState {
const PAGE_SIZE = 200; const PAGE_SIZE = 200;
const AI_CAPTIONS_ENABLED_KEY = "phokus.aiCaptionsEnabled"; const AI_CAPTIONS_ENABLED_KEY = "phokus.aiCaptionsEnabled";
const SIMILAR_DISTANCE_THRESHOLD = 0.24;
let galleryRequestToken = 0;
let similarRequestToken = 0;
let tagCloudRequestToken = 0;
let exploreTagRequestToken = 0;
function initialAiCaptionsEnabled(): boolean { function initialAiCaptionsEnabled(): boolean {
if (typeof window === "undefined") return false; if (typeof window === "undefined") return false;
@@ -312,19 +388,71 @@ function matchesSearch(image: ImageRecord, search: string): boolean {
return image.filename.toLowerCase().includes(search.toLowerCase()); return image.filename.toLowerCase().includes(search.toLowerCase());
} }
function isDerivedCollectionTitle(collectionTitle: string | null): boolean {
return collectionTitle !== null;
}
export function parseSearchValue(search: string): ParsedSearch {
if (!search.trim()) {
return { mode: "filename", query: "", prefix: null };
}
const slashPrefix = search.match(/^\/([a-z])(?:\s|$)/i);
if (slashPrefix) {
const rawPrefix = slashPrefix[1].toLowerCase();
const query = search.length > 3 ? search.slice(3) : "";
if (rawPrefix === "s") {
return { mode: "semantic", query, prefix: "/s" };
}
if (rawPrefix === "t") {
return { mode: "tag", query, prefix: "/t" };
}
return { mode: "filename", query, prefix: rawPrefix === "f" ? "/f" : null };
}
const trimmed = search.trim();
const match = trimmed.match(/^([a-z]):\s*(.*)$/i);
if (!match) {
return { mode: "filename", query: trimmed, prefix: null };
}
const rawPrefix = match[1].toLowerCase();
const query = match[2].trim();
if (rawPrefix === "s") {
return { mode: "semantic", query, prefix: "s:" };
}
if (rawPrefix === "t") {
return { mode: "tag", query, prefix: "t:" };
}
return { mode: "filename", query, prefix: rawPrefix === "f" ? "f:" : null };
}
export function searchModeLabel(mode: SearchCommand): string {
switch (mode) {
case "semantic":
return "Semantic Search";
case "tag":
return "Tag Search";
default:
return "Filename Search";
}
}
function matchesFilters( function matchesFilters(
image: ImageRecord, image: ImageRecord,
selectedFolderId: number | null, selectedFolderId: number | null,
mediaFilter: MediaFilter, mediaFilter: MediaFilter,
favoritesOnly: boolean, favoritesOnly: boolean,
minimumRating: number,
failedEmbeddingsOnly: boolean, failedEmbeddingsOnly: boolean,
search: string, search: string,
): boolean { ): boolean {
const matchesFolder = selectedFolderId === null || image.folder_id === selectedFolderId; const matchesFolder = selectedFolderId === null || image.folder_id === selectedFolderId;
const matchesMedia = mediaFilter === "all" || image.media_kind === mediaFilter; const matchesMedia = mediaFilter === "all" || image.media_kind === mediaFilter;
const matchesFavorite = !favoritesOnly || image.favorite; const matchesFavorite = !favoritesOnly || image.favorite;
const matchesRating = image.rating >= minimumRating;
const matchesFailedEmbedding = !failedEmbeddingsOnly || image.embedding_status === "failed"; const matchesFailedEmbedding = !failedEmbeddingsOnly || image.embedding_status === "failed";
return matchesFolder && matchesMedia && matchesFavorite && matchesFailedEmbedding && matchesSearch(image, search); return matchesFolder && matchesMedia && matchesFavorite && matchesRating && matchesFailedEmbedding && matchesSearch(image, search);
} }
function compareNullableNumber(a: number | null, b: number | null): number { function compareNullableNumber(a: number | null, b: number | null): number {
@@ -349,6 +477,10 @@ function compareImages(a: ImageRecord, b: ImageRecord, sort: SortOrder): number
return compareNullableNumber(a.file_size, b.file_size); return compareNullableNumber(a.file_size, b.file_size);
case "size_desc": case "size_desc":
return compareNullableNumber(b.file_size, a.file_size); return compareNullableNumber(b.file_size, a.file_size);
case "rating_asc":
return compareNullableNumber(a.rating, b.rating);
case "rating_desc":
return compareNullableNumber(b.rating, a.rating);
case "duration_asc": case "duration_asc":
return compareNullableNumber(a.duration_ms, b.duration_ms); return compareNullableNumber(a.duration_ms, b.duration_ms);
case "duration_desc": case "duration_desc":
@@ -424,17 +556,25 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
sort: "date_desc", sort: "date_desc",
mediaFilter: "all", mediaFilter: "all",
favoritesOnly: false, favoritesOnly: false,
minimumRating: 0,
failedEmbeddingsOnly: false, failedEmbeddingsOnly: false,
zoomPreset: "comfortable", zoomPreset: "comfortable",
selectedImage: null, selectedImage: null,
collectionTitle: null, collectionTitle: null,
similarSourceImageId: null, similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false, similarHasMore: false,
similarScope: "all_media",
similarFolderId: null,
galleryScrollResetKey: 0, galleryScrollResetKey: 0,
activeView: "gallery", activeView: "gallery",
exploreMode: "visual",
tagCloudEntries: [], tagCloudEntries: [],
tagCloudLoading: false, tagCloudLoading: false,
tagCloudFolderId: undefined, tagCloudFolderId: undefined,
exploreTagEntries: [],
exploreTagLoading: false,
exploreTagsFolderId: undefined,
indexingProgress: {}, indexingProgress: {},
mediaJobProgress: {}, mediaJobProgress: {},
cacheDir: "", cacheDir: "",
@@ -448,6 +588,8 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
captionDetail: "paragraph", captionDetail: "paragraph",
aiCaptionsEnabled: initialAiCaptionsEnabled(), aiCaptionsEnabled: initialAiCaptionsEnabled(),
settingsOpen: false, settingsOpen: false,
taggingQueueScope: "all",
taggingQueueFolderIds: [],
taggerModelStatus: null, taggerModelStatus: null,
taggerModelPreparing: false, taggerModelPreparing: false,
@@ -455,14 +597,33 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
taggerModelProgress: null, taggerModelProgress: null,
taggerAcceleration: "auto", taggerAcceleration: "auto",
taggerThreshold: 0.35, taggerThreshold: 0.35,
taggerBatchSize: 8,
taggerRuntimeProbe: null, taggerRuntimeProbe: null,
taggerRuntimeChecking: false, taggerRuntimeChecking: false,
duplicateGroups: [],
duplicateScanning: false,
duplicateScanProgress: null,
duplicateSelectedIds: new Set(),
duplicateLastScanned: null,
setCacheDir: (cacheDir) => set({ cacheDir }), setCacheDir: (cacheDir) => set({ cacheDir }),
loadFolders: async () => { loadFolders: async () => {
const folders = await invoke<Folder[]>("get_folders"); const folders = await invoke<Folder[]>("get_folders");
set({ folders }); set((state) => {
const folderIds = new Set(folders.map((folder) => folder.id));
const nextSelected = state.taggingQueueFolderIds.filter((folderId) => folderIds.has(folderId));
return {
folders,
taggingQueueFolderIds:
nextSelected.length > 0
? nextSelected
: state.taggingQueueScope === "selected" && folders.length > 0
? [folders[0].id]
: nextSelected,
};
});
}, },
loadBackgroundJobProgress: async () => { loadBackgroundJobProgress: async () => {
@@ -507,29 +668,64 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
}, },
loadImages: async (reset = false) => { loadImages: async (reset = false) => {
const { selectedFolderId, search, searchMode, sort, loadedCount, mediaFilter, favoritesOnly, failedEmbeddingsOnly } = get(); const { selectedFolderId, search, sort, loadedCount, mediaFilter, favoritesOnly, minimumRating, failedEmbeddingsOnly } = get();
const parsedSearch = parseSearchValue(search);
const requestToken = ++galleryRequestToken;
set({ loadingImages: true, imageLoadError: null }); set({ loadingImages: true, imageLoadError: null });
try { try {
if (searchMode === "semantic" && search.trim()) { if (parsedSearch.mode === "semantic" && parsedSearch.query) {
const images = await invoke<ImageRecord[]>("semantic_search_images", { const images = await invoke<ImageRecord[]>("semantic_search_images", {
params: { params: {
query: search, query: parsedSearch.query,
folder_id: selectedFolderId, folder_id: selectedFolderId,
media_kind: mediaFilter === "all" ? null : mediaFilter, media_kind: mediaFilter === "all" ? null : mediaFilter,
favorites_only: favoritesOnly, favorites_only: favoritesOnly,
rating_min: minimumRating > 0 ? minimumRating : null,
limit: PAGE_SIZE, limit: PAGE_SIZE,
}, },
}); });
if (requestToken !== galleryRequestToken) return;
set({ set({
images, images,
totalImages: images.length, totalImages: images.length,
loadedCount: images.length, loadedCount: images.length,
loadingImages: false, loadingImages: false,
collectionTitle: `Semantic search: ${search}`, collectionTitle: `Semantic search: ${parsedSearch.query}`,
selectedFolderId,
similarSourceImageId: null, similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false, similarHasMore: false,
similarFolderId: null,
});
return;
}
if (parsedSearch.mode === "tag" && parsedSearch.query) {
const images = await invoke<ImageRecord[]>("search_images_by_tag", {
params: {
query: parsedSearch.query,
folder_id: selectedFolderId,
media_kind: mediaFilter === "all" ? null : mediaFilter,
favorites_only: favoritesOnly,
rating_min: minimumRating > 0 ? minimumRating : null,
limit: PAGE_SIZE,
},
});
if (requestToken !== galleryRequestToken) return;
set({
images,
totalImages: images.length,
loadedCount: images.length,
loadingImages: false,
collectionTitle: `Tag search: ${parsedSearch.query}`,
selectedFolderId,
similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false,
similarFolderId: null,
}); });
return; return;
} }
@@ -543,9 +739,10 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
}>("get_images", { }>("get_images", {
params: { params: {
folder_id: selectedFolderId, folder_id: selectedFolderId,
search: search || null, search: parsedSearch.query || null,
media_kind: mediaFilter === "all" ? null : mediaFilter, media_kind: mediaFilter === "all" ? null : mediaFilter,
favorites_only: favoritesOnly, favorites_only: favoritesOnly,
rating_min: minimumRating > 0 ? minimumRating : null,
embedding_failed_only: failedEmbeddingsOnly, embedding_failed_only: failedEmbeddingsOnly,
sort, sort,
offset, offset,
@@ -553,6 +750,7 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
}, },
}); });
if (requestToken !== galleryRequestToken) return;
set((state) => ({ set((state) => ({
images: reset ? result.images : [...state.images, ...result.images], images: reset ? result.images : [...state.images, ...result.images],
totalImages: result.total, totalImages: result.total,
@@ -560,20 +758,24 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
loadingImages: false, loadingImages: false,
collectionTitle: reset ? null : state.collectionTitle, collectionTitle: reset ? null : state.collectionTitle,
similarSourceImageId: null, similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false, similarHasMore: false,
similarFolderId: null,
})); }));
} catch (error) { } catch (error) {
if (requestToken !== galleryRequestToken) return;
console.error("Failed to load media:", error); console.error("Failed to load media:", error);
set({ loadingImages: false, imageLoadError: String(error) }); set({ loadingImages: false, imageLoadError: String(error) });
} }
}, },
loadMoreImages: async () => { loadMoreImages: async () => {
const { loadedCount, totalImages, loadingImages, collectionTitle, similarSourceImageId, similarHasMore, selectedFolderId } = get(); const { loadedCount, totalImages, loadingImages, collectionTitle, similarSourceImageId, similarHasMore, similarFolderId } = get();
if (loadingImages || loadedCount >= totalImages) return; if (loadingImages || loadedCount >= totalImages) return;
if (collectionTitle === "Explore Cluster") return;
if (collectionTitle === "Similar Images" && similarSourceImageId !== null) { if (collectionTitle === "Similar Images" && similarSourceImageId !== null) {
if (!similarHasMore) return; if (!similarHasMore) return;
await get().loadSimilarImages(similarSourceImageId, selectedFolderId, false); await get().loadSimilarImages(similarSourceImageId, similarFolderId, false);
return; return;
} }
await get().loadImages(false); await get().loadImages(false);
@@ -614,6 +816,11 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
void get().loadImages(true); void get().loadImages(true);
}, },
setMinimumRating: (minimumRating) => {
set({ minimumRating, images: [], loadedCount: 0, collectionTitle: null, similarSourceImageId: null, similarHasMore: false, imageLoadError: null });
void get().loadImages(true);
},
setFailedEmbeddingsOnly: (failedEmbeddingsOnly) => { setFailedEmbeddingsOnly: (failedEmbeddingsOnly) => {
set({ failedEmbeddingsOnly, images: [], loadedCount: 0, collectionTitle: null, similarSourceImageId: null, similarHasMore: false, imageLoadError: null }); set({ failedEmbeddingsOnly, images: [], loadedCount: 0, collectionTitle: null, similarSourceImageId: null, similarHasMore: false, imageLoadError: null });
void get().loadImages(true); void get().loadImages(true);
@@ -626,59 +833,148 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
setView: (activeView) => set({ activeView }), setView: (activeView) => set({ activeView }),
setExploreMode: (exploreMode) => set({ exploreMode }),
loadTagCloud: async () => { loadTagCloud: async () => {
const { selectedFolderId, tagCloudFolderId, tagCloudLoading } = get(); const { selectedFolderId, tagCloudFolderId, tagCloudLoading } = get();
// Skip if already loaded for this folder and not currently loading // Skip if already loaded for this folder and not currently loading
if (!tagCloudLoading && tagCloudFolderId !== undefined && tagCloudFolderId === selectedFolderId) { if (!tagCloudLoading && tagCloudFolderId !== undefined && tagCloudFolderId === selectedFolderId) {
return; return;
} }
const requestToken = ++tagCloudRequestToken;
set({ tagCloudLoading: true, tagCloudFolderId: selectedFolderId }); set({ tagCloudLoading: true, tagCloudFolderId: selectedFolderId });
try { try {
const entries = await invoke<TagCloudEntry[]>("get_tag_cloud", { const entries = await invoke<TagCloudEntry[]>("get_tag_cloud", {
folderId: selectedFolderId, folderId: selectedFolderId,
}); });
if (requestToken !== tagCloudRequestToken) return;
set({ tagCloudEntries: entries, tagCloudLoading: false }); set({ tagCloudEntries: entries, tagCloudLoading: false });
} catch (error) { } catch (error) {
if (requestToken !== tagCloudRequestToken) return;
console.error("Failed to load tag cloud:", error); console.error("Failed to load tag cloud:", error);
set({ tagCloudLoading: false }); set({ tagCloudLoading: false });
} }
}, },
searchByTag: (imageId) => { loadExploreTags: async () => {
const { selectedFolderId } = get(); const { selectedFolderId, exploreTagsFolderId, exploreTagLoading } = get();
set((state) => ({ activeView: "gallery", images: [], loadedCount: 0, loadingImages: true, collectionTitle: "Similar Images", imageLoadError: null, galleryScrollResetKey: state.galleryScrollResetKey + 1 })); if (!exploreTagLoading && exploreTagsFolderId !== undefined && exploreTagsFolderId === selectedFolderId) {
void get().loadSimilarImages(imageId, selectedFolderId); return;
}
const requestToken = ++exploreTagRequestToken;
set({ exploreTagLoading: true, exploreTagsFolderId: selectedFolderId });
try {
const entries = await invoke<ExploreTagEntry[]>("get_explore_tags", {
params: { folder_id: selectedFolderId, limit: 48 },
});
if (requestToken !== exploreTagRequestToken) return;
set({ exploreTagEntries: entries, exploreTagLoading: false });
} catch (error) {
if (requestToken !== exploreTagRequestToken) return;
console.error("Failed to load explore tags:", error);
set({ exploreTagLoading: false });
}
}, },
loadSimilarImages: async (imageId, folderId = get().selectedFolderId, reset = true) => { showVisualCluster: async (imageIds) => {
const requestedLimit = reset ? PAGE_SIZE : get().loadedCount + PAGE_SIZE; const requestToken = ++similarRequestToken;
set((state) => ({ set((state) => ({
images: reset ? [] : get().images, activeView: "gallery",
loadedCount: reset ? 0 : get().loadedCount, search: "",
images: [],
totalImages: imageIds.length,
loadedCount: 0,
loadingImages: true,
collectionTitle: "Explore Cluster",
imageLoadError: null,
similarSourceImageId: null,
similarSourceFolderId: null,
similarHasMore: false,
similarFolderId: null,
galleryScrollResetKey: state.galleryScrollResetKey + 1,
}));
try {
const images = await invoke<ImageRecord[]>("get_images_by_ids", {
params: { image_ids: imageIds },
});
if (requestToken !== similarRequestToken) return;
set({
images,
totalImages: images.length,
loadedCount: images.length,
loadingImages: false,
imageLoadError: null,
collectionTitle: "Explore Cluster",
});
} catch (error) {
if (requestToken !== similarRequestToken) return;
set({
images: [],
totalImages: 0,
loadedCount: 0,
loadingImages: false,
imageLoadError: String(error),
collectionTitle: "Explore Cluster",
});
}
},
searchForTag: (tag) => {
set({ activeView: "gallery", search: `/t ${tag}`, images: [], loadedCount: 0, collectionTitle: null, similarSourceImageId: null, similarHasMore: false, similarFolderId: null, imageLoadError: null });
void get().loadImages(true);
},
loadSimilarImages: async (imageId, folderId = get().selectedFolderId, reset = true, sourceFolderId = folderId ?? null) => {
const requestToken = ++similarRequestToken;
const offset = reset ? 0 : get().loadedCount;
const similarScope = folderId === null ? "all_media" : "current_folder";
set((state) => ({
images: reset ? [] : state.images,
loadedCount: reset ? 0 : state.loadedCount,
loadingImages: true, loadingImages: true,
collectionTitle: "Similar Images", collectionTitle: "Similar Images",
imageLoadError: null, imageLoadError: null,
similarSourceImageId: imageId, similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarFolderId: folderId ?? null,
similarScope,
galleryScrollResetKey: reset ? state.galleryScrollResetKey + 1 : state.galleryScrollResetKey, galleryScrollResetKey: reset ? state.galleryScrollResetKey + 1 : state.galleryScrollResetKey,
})); }));
try {
const images = await invoke<ImageRecord[]>("find_similar_images", { try {
params: { image_id: imageId, folder_id: folderId ?? null, limit: requestedLimit }, const result = await invoke<SimilarImagesPage>("find_similar_images", {
}); params: {
const hasMore = images.length >= requestedLimit; image_id: imageId,
set({ folder_id: folderId ?? null,
images, offset,
totalImages: hasMore ? images.length + PAGE_SIZE : images.length, limit: PAGE_SIZE,
loadedCount: images.length, threshold: SIMILAR_DISTANCE_THRESHOLD,
loadingImages: false, },
imageLoadError: null, });
collectionTitle: "Similar Images",
similarSourceImageId: imageId, if (requestToken !== similarRequestToken) return;
similarHasMore: hasMore,
selectedFolderId: folderId ?? null, set((state) => {
selectedImage: reset ? null : get().selectedImage, const nextImages = reset ? result.images : [...state.images, ...result.images];
const nextLoadedCount = nextImages.length;
return {
images: nextImages,
totalImages: result.has_more ? nextLoadedCount + 1 : nextLoadedCount,
loadedCount: nextLoadedCount,
loadingImages: false,
imageLoadError: null,
collectionTitle: "Similar Images",
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarHasMore: result.has_more,
similarFolderId: folderId ?? null,
similarScope,
selectedImage: reset ? null : state.selectedImage,
};
}); });
} catch (error) { } catch (error) {
if (requestToken !== similarRequestToken) return;
console.error("Failed to load similar images:", error); console.error("Failed to load similar images:", error);
set({ set({
images: [], images: [],
@@ -688,13 +984,89 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
imageLoadError: String(error), imageLoadError: String(error),
collectionTitle: "Similar Images", collectionTitle: "Similar Images",
similarSourceImageId: imageId, similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarHasMore: false, similarHasMore: false,
selectedFolderId: folderId ?? null, similarFolderId: folderId ?? null,
similarScope,
selectedImage: null, selectedImage: null,
}); });
} }
}, },
loadSimilarByRegion: async (imageId, crop, folderId = get().selectedFolderId, sourceFolderId = folderId ?? null) => {
const requestToken = ++similarRequestToken;
const similarScope = folderId === null ? "all_media" : "current_folder";
set((state) => ({
images: [],
loadedCount: 0,
loadingImages: true,
collectionTitle: "Region Search Results",
imageLoadError: null,
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarFolderId: folderId ?? null,
similarScope,
galleryScrollResetKey: state.galleryScrollResetKey + 1,
selectedImage: null,
}));
try {
const result = await invoke<SimilarImagesPage>("find_similar_by_region", {
params: {
image_id: imageId,
crop_x: crop.x,
crop_y: crop.y,
crop_w: crop.w,
crop_h: crop.h,
folder_id: folderId ?? null,
offset: 0,
limit: PAGE_SIZE,
},
});
if (requestToken !== similarRequestToken) return;
set({
images: result.images,
totalImages: result.has_more ? result.images.length + 1 : result.images.length,
loadedCount: result.images.length,
loadingImages: false,
imageLoadError: null,
collectionTitle: "Region Search Results",
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarHasMore: result.has_more,
similarFolderId: folderId ?? null,
similarScope,
});
} catch (error) {
if (requestToken !== similarRequestToken) return;
console.error("Failed to load region search results:", error);
set({
images: [],
totalImages: 0,
loadedCount: 0,
loadingImages: false,
imageLoadError: String(error),
collectionTitle: "Region Search Results",
similarSourceImageId: imageId,
similarSourceFolderId: sourceFolderId,
similarHasMore: false,
similarFolderId: folderId ?? null,
similarScope,
selectedImage: null,
});
}
},
setSimilarScope: (similarScope) => {
set({ similarScope });
const { similarSourceImageId, similarSourceFolderId, selectedFolderId } = get();
if (similarSourceImageId === null) return;
const folderId = similarScope === "current_folder" ? (similarSourceFolderId ?? selectedFolderId) : null;
void get().loadSimilarImages(similarSourceImageId, folderId, true, similarSourceFolderId);
},
suggestImageTags: async (imageId) => { suggestImageTags: async (imageId) => {
return invoke<string[]>("suggest_image_tags", { return invoke<string[]>("suggest_image_tags", {
params: { image_id: imageId, limit: 2 }, params: { image_id: imageId, limit: 2 },
@@ -833,6 +1205,27 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
setSettingsOpen: (settingsOpen) => set({ settingsOpen }), setSettingsOpen: (settingsOpen) => set({ settingsOpen }),
setTaggingQueueScope: (taggingQueueScope) => {
set((state) => ({
taggingQueueScope,
taggingQueueFolderIds:
taggingQueueScope === "selected" && state.taggingQueueFolderIds.length === 0 && state.folders.length > 0
? [state.folders[0].id]
: state.taggingQueueFolderIds,
}));
},
toggleTaggingQueueFolder: (folderId) => {
set((state) => {
const next = state.taggingQueueFolderIds.includes(folderId)
? state.taggingQueueFolderIds.filter((id) => id !== folderId)
: [...state.taggingQueueFolderIds, folderId].sort((a, b) => a - b);
return { taggingQueueFolderIds: next };
});
},
setTaggingQueueFolderIds: (taggingQueueFolderIds) => set({ taggingQueueFolderIds }),
loadTaggerModelStatus: async () => { loadTaggerModelStatus: async () => {
try { try {
const taggerModelStatus = await invoke<TaggerModelStatus>("get_tagger_model_status"); const taggerModelStatus = await invoke<TaggerModelStatus>("get_tagger_model_status");
@@ -874,6 +1267,22 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
set({ taggerThreshold }); set({ taggerThreshold });
}, },
loadTaggerBatchSize: async () => {
try {
const taggerBatchSize = await invoke<number>("get_tagger_batch_size");
set({ taggerBatchSize });
} catch (error) {
set({ taggerModelError: String(error) });
}
},
setTaggerBatchSize: async (batchSize) => {
const taggerBatchSize = await invoke<number>("set_tagger_batch_size", {
params: { batch_size: batchSize },
});
set({ taggerBatchSize });
},
prepareTaggerModel: async () => { prepareTaggerModel: async () => {
set({ taggerModelPreparing: true, taggerModelError: null, taggerModelProgress: null }); set({ taggerModelPreparing: true, taggerModelError: null, taggerModelProgress: null });
try { try {
@@ -912,6 +1321,14 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
return queued; return queued;
}, },
queueTaggingJobsForFolders: async (folderIds) => {
const queued = await invoke<number>("queue_tagging_jobs", {
params: { folder_id: null, folder_ids: folderIds, image_id: null },
});
await get().loadBackgroundJobProgress();
return queued;
},
queueTaggingForImage: async (imageId) => { queueTaggingForImage: async (imageId) => {
const queued = await invoke<number>("queue_tagging_jobs", { const queued = await invoke<number>("queue_tagging_jobs", {
params: { folder_id: null, image_id: imageId }, params: { folder_id: null, image_id: imageId },
@@ -928,6 +1345,14 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
return cleared; return cleared;
}, },
clearTaggingJobsForFolders: async (folderIds) => {
const cleared = await invoke<number>("clear_tagging_jobs", {
params: { folder_id: null, folder_ids: folderIds },
});
await get().loadBackgroundJobProgress();
return cleared;
},
getImageTags: async (imageId) => { getImageTags: async (imageId) => {
return invoke<ImageTag[]>("get_image_tags", { return invoke<ImageTag[]>("get_image_tags", {
params: { image_id: imageId }, params: { image_id: imageId },
@@ -946,6 +1371,73 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
}); });
}, },
loadDuplicateScanCache: async (folderId = null) => {
interface CacheResult { groups: DuplicateGroup[]; scanned_at: number }
const cached = await invoke<CacheResult | null>("load_duplicate_scan_cache", { folderId: folderId ?? null });
if (cached) {
set({ duplicateGroups: cached.groups, duplicateLastScanned: cached.scanned_at });
}
},
scanDuplicates: async (folderId = null) => {
const { listen } = await import("@tauri-apps/api/event");
set({ duplicateScanning: true, duplicateGroups: [], duplicateScanProgress: null, duplicateSelectedIds: new Set() });
const unlisten = await listen<[number, number]>("duplicate_scan_progress", (event) => {
const [scanned, total] = event.payload;
set({ duplicateScanProgress: { scanned, total } });
});
try {
const groups = await invoke<DuplicateGroup[]>("find_duplicates", { folderId: folderId ?? null });
set({ duplicateGroups: groups, duplicateLastScanned: Math.floor(Date.now() / 1000) });
} finally {
unlisten();
set({ duplicateScanning: false });
}
},
toggleDuplicateSelected: (imageId) => {
set((state) => {
const next = new Set(state.duplicateSelectedIds);
if (next.has(imageId)) next.delete(imageId);
else next.add(imageId);
return { duplicateSelectedIds: next };
});
},
selectAllDuplicates: (imageIds) => {
set((state) => {
const next = new Set(state.duplicateSelectedIds);
for (const id of imageIds) next.add(id);
return { duplicateSelectedIds: next };
});
},
selectKeepFirstAllGroups: () => {
const { duplicateGroups } = get();
const toMark = new Set<number>();
for (const group of duplicateGroups) {
for (const img of group.images.slice(1)) toMark.add(img.id);
}
set({ duplicateSelectedIds: toMark });
},
clearDuplicateSelection: () => set({ duplicateSelectedIds: new Set() }),
deleteSelectedDuplicates: async () => {
const { duplicateSelectedIds } = get();
const ids = Array.from(duplicateSelectedIds);
if (ids.length === 0) return 0;
const deleted = await invoke<number>("delete_images_from_disk", { params: { image_ids: ids } });
// Remove deleted images from groups and drop now-trivial groups
set((state) => ({
duplicateSelectedIds: new Set(),
duplicateGroups: state.duplicateGroups
.map((g) => ({ ...g, images: g.images.filter((img) => !duplicateSelectedIds.has(img.id)) }))
.filter((g) => g.images.length > 1),
}));
return deleted;
},
retryFailedEmbeddings: async (folderId) => { retryFailedEmbeddings: async (folderId) => {
await invoke("retry_failed_embeddings", { params: { folder_id: folderId } }); await invoke("retry_failed_embeddings", { params: { folder_id: folderId } });
await get().loadBackgroundJobProgress(); await get().loadBackgroundJobProgress();
@@ -979,7 +1471,9 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
if (progress.done) { if (progress.done) {
void get().loadFolders(); void get().loadFolders();
void get().loadBackgroundJobProgress(); void get().loadBackgroundJobProgress();
void get().loadImages(true); if (get().activeView !== "explore" && !isDerivedCollectionTitle(get().collectionTitle)) {
void get().loadImages(true);
}
setTimeout(() => { setTimeout(() => {
set((state) => { set((state) => {
@@ -1019,12 +1513,17 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
const batch = event.payload; const batch = event.payload;
set((state) => { set((state) => {
if (isDerivedCollectionTitle(state.collectionTitle) || state.activeView === "explore") {
return state;
}
const visibleImages = batch.images.filter((image) => const visibleImages = batch.images.filter((image) =>
matchesFilters( matchesFilters(
image, image,
state.selectedFolderId, state.selectedFolderId,
state.mediaFilter, state.mediaFilter,
state.favoritesOnly, state.favoritesOnly,
state.minimumRating,
state.failedEmbeddingsOnly, state.failedEmbeddingsOnly,
state.search, state.search,
), ),
@@ -1049,12 +1548,21 @@ export const useGalleryStore = create<GalleryState>((set, get) => ({
const batch = event.payload; const batch = event.payload;
set((state) => { set((state) => {
if (isDerivedCollectionTitle(state.collectionTitle) || state.activeView === "explore") {
const selectedImage =
state.selectedImage && batch.images.some((image) => image.id === state.selectedImage?.id)
? batch.images.find((image) => image.id === state.selectedImage?.id) ?? state.selectedImage
: state.selectedImage;
return { selectedImage };
}
const visibleImages = batch.images.filter((image) => const visibleImages = batch.images.filter((image) =>
matchesFilters( matchesFilters(
image, image,
state.selectedFolderId, state.selectedFolderId,
state.mediaFilter, state.mediaFilter,
state.favoritesOnly, state.favoritesOnly,
state.minimumRating,
state.failedEmbeddingsOnly, state.failedEmbeddingsOnly,
state.search, state.search,
), ),