storage.rs: thumbnail worker clamping across parallelism levels, the
adaptive-profile EMA transitions, and UNC-path fallback detection.
indexer.rs: supported-media extension matching plus media-kind and MIME
mapping. thumbnail.rs: fit_dimensions aspect-ratio math with extreme
ratios and is_jpeg extension checks. ai_tag_filter.rs: padding and
mixed-separator normalization edge cases.
Worker pause states can optionally survive app restarts. A new toggle in
Settings saves the current pause map to settings/worker_pauses.json; on
startup lib.rs restores it before workers are spawned. Backend: new
snapshot/replace helpers in indexer.rs, persist functions in commands.rs
(get/set_worker_pauses_persist). Frontend: workerPausesPersist store
field, load/setWorkerPausesPersist actions, toggle in SettingsModal.
Introduces a selectable tagging model so Phokus isn't locked to the
anime-leaning WD tagger. JoyTag uses the Danbooru schema but generalizes to
photographic content and is strong on NSFW concepts — a better fit for a photo
manager while keeping the explicitness range.
- `TaggerModel` enum (wd | joytag) persisted as a `tagger_model` setting;
`model_dir`, status, and download now follow the active model (per-model
repo/dir/file list). WD stays the default.
- `Tagger` trait implemented by `WdTagger` and the new `JoyTagger`;
`create_active_tagger` builds the selected one. The worker holds
`Box<dyn Tagger>` and rebuilds on model change (TAGGER_SESSION_DIRTY).
- Shared `assemble_batch` skeleton (pack -> one forward pass -> per-image
fallback, results in input order); both providers and the shared
decode/pad/resize are de-duped onto common helpers.
- JoyTag specifics: NCHW + RGB + CLIP-normalized input (vs WD's NHWC/BGR/raw),
flat top_tags.txt labels, logits -> sigmoid -> threshold (default 0.4). It has
no native rating, so the explicitness rating is derived from its NSFW tags.
- Tags are attributed to the model that produced them (ai_tagger_model),
via Tagger::model_name, instead of a hardcoded WD constant.
- New get/set_tagger_model commands.
Backend only; the Settings model picker and end-to-end testing against the
JoyTag model files come next.
The tagging worker claimed a batch from the DB but ran the model one image
at a time, so `tagger_batch_size` had no effect on inference. Batching the
whole claim into a single DirectML forward pass fixed that but introduced a
worse problem: on a shared GPU each wide dispatch locks the device (and the
WebView2 compositor with it) for 1-3.7s, freezing the entire app while
tagging runs — worst of all on first launch with a cold graph compile.
The WD model is compute-bound here (~50-230ms/image of actual GPU work), so
a wide batch buys almost no throughput; it only lengthens each uninterruptible
GPU lock. So decouple DB claim size from GPU granularity: claim
`tagger_batch_size` for DB efficiency, but feed the GPU in TAGGER_INFER_CHUNK
(4) images per forward pass with a brief yield between chunks. Each dispatch
now lasts ~200-900ms, the UI gets windows to paint, peak decode memory is
bounded, and the cold compile is for a smaller shape. As a bonus the wide-batch
throughput spikes disappear — steady ~1.6-1.8s/16 vs the old 0.8-3.7s swings.
- run_batch: parallel (rayon) decode + single forward pass per chunk, with
per-image fallback and decode failures kept attached to their input slot.
- Worker iterates source_paths.chunks(TAGGER_INFER_CHUNK), yielding 40ms
between chunks; outputs zip back to jobs 1:1, write tx unchanged.
- Remove now-dead WdTagger::run() (fallback uses infer_one directly).
Add dominant-color palette extraction, storage, filtering, and startup backfill so the gallery and Timeline can be filtered from the toolbar color picker.
Introduce a reusable tooltip component and migrate the color filter, update indicator, and gallery filename hover affordances to it.
Route AVIF thumbnail generation through the bundled FFmpeg path instead of the Rust image decoder, avoiding unsupported-format failures without requiring system dav1d dependencies.
Requeue existing AVIF jobs that previously failed with unsupported-format errors and feed generated JPEG derivatives to embedding/tagging preprocessing while leaving lightbox display on the original AVIF file.
Derive default implementations for captioner and tagger option enums, simplify sorting and progress multiple checks, and remove redundant iterator conversions.
Drops and recreates the sqlite-vec tables at the current model dimension, then
re-queues every image for embedding. Fixes "dimension mismatch" search errors
that occur when the vector table was built for a different model/dimension (e.g.
after experimenting with 768-dim models against this 512-dim build).
The rebuild runs under the embedding worker's DB write lock and resets the job
queue inside a single transaction, so it can't interleave with an in-flight
embedding batch (per code review).
Timeline:
- Add a right-edge scrubber (year labels + month dots) that jumps to any
period; runs in the same direction as the scrolled content
- Load the full filtered set in Timeline view so the scrubber spans the whole
library instead of just the first page
Gallery & UI:
- Virtualise the gallery grid
- Folder reordering in the sidebar (drag + persisted sort_order) and a themed
dropdown component
- Subtle Light theme contrast fixes for toggles, secondary buttons and the
update toast
Embedding workers now defer video jobs without a thumbnail at claim time and
requeue any previously-failed deferred jobs on startup, so videos no longer
churn through failed embeddings.
QoL polish across BackgroundTasks, DuplicateFinder, Lightbox and VideoPlayer.
Phase 5 of the 0.1.0 release prep:
- FFmpeg no longer blocks startup (or crashes the app offline): provisioning
runs in a background thread emitting throttled ffmpeg-progress events,
with installed/downloading/failed state queryable via get_ffmpeg_status
and a retry command
- video jobs are invisible to claiming and tier-priority checks until FFmpeg
is ready, so they wait as pending without failing — and without starving
image embeddings/tagging (include_videos gating in db.rs)
- 7-step show-don't-tell onboarding wizard: welcome + live FFmpeg progress,
real first-folder picker, faked animating pipeline bar, placeholder
gallery tiles, cycling search-syntax demo, views overview, and an AI
features step with a real opt-in tagger download
- skippable at every point (Escape included), persisted via
settings/onboarding_completed.txt, re-runnable from Settings > General,
which also gains an FFmpeg status/retry row
Auto-fix pass (uninlined format args, needless borrows, to_vec,
size_of_val, map_err->inspect_err) plus hand fixes: fully annotated the
sqlite-vec transmute (c_char for cross-platform correctness), type alias
for the duplicate-scan tuple, slice signatures over &mut Vec/&PathBuf,
and #[allow(dead_code)] documenting the intentionally dormant caption
worker. The CI clippy step now fails on any new warning.
Phase 4 of the 0.1.0 release prep:
- tauri-plugin-log: rotating file log (5 MB, Info) in the app log dir plus
stdout; all 54 backend println!/eprintln! sites migrated to log macros so
release builds finally produce diagnostics
- tauri-plugin-single-instance (registered first): second launches focus the
existing window instead of racing the same SQLite DB with a second worker
fleet; tauri-plugin-window-state persists window geometry
- real CSP replacing null (scripts self-only, media/images via the asset
protocol, IPC endpoints whitelisted, object-src none) with a devCsp
variant for Vite HMR
- asset protocol scope narrowed from '**' to thumbnails statically plus
per-folder runtime grants (setup, add_folder, update_folder_path)
Panic audit (plan item) concluded with no changes: worker loops already
catch and log all Result errors; remaining unwraps are startup fail-fast,
poisoned-lock convention, infallible-by-construction, or test code.
Workers now form a priority pipeline (thumbnails -> metadata ->
embeddings -> tagging): a worker only claims jobs when no
higher-priority tier has claimable work, so stages drain one at a time
at full speed instead of all workers contending for CPU (shared rayon
pool), disk, and the DB writer simultaneously.
Each tier gate is a single read-only EXISTS query per idle tick using
the same exclusion set the corresponding claim would (that worker''s
paused folders plus actively-indexing folders), so paused or mid-scan
work never blocks lower tiers, and failed jobs stop gating once they
leave pending. Deferred workers wake within one backoff interval
(250-750ms) of the higher tier draining.
A watcher-driven trickle of new files briefly pauses lower tiers by
design. A user-initiated tag run during a large embedding backlog
waits for it; pausing embeddings per folder overrides if tags are
wanted first.
The metadata worker now sleeps its 250ms backoff only when the queue
is empty or a batch errored, claiming the next batch immediately while
work is pending. Each ffprobe result is committed and emitted
individually (in its own transaction) instead of after the whole
batch, so video metadata progress moves steadily rather than stalling
for up to 16 sequential probes.
Embedding and tagging claims now exclude actively-indexing folders,
matching the thumbnail and metadata workers. Previously the embedding
worker started processing a folder while it was still being scanned,
competing with the scanner for CPU (shared rayon pool), disk, and the
DB writer — slowing scans and skewing the adaptive storage profile
toward Conservative. Video embedding jobs claimed mid-scan also
fail-fasted pointlessly since their thumbnails are deferred until
indexing completes; the existing backfill at scan end covers requeue.
- Embedding and tagging workers now sleep their backoff interval
(500ms/750ms) only when the queue is empty, the model is not ready,
or a batch errored — previously every batch paid the sleep even with
work pending.
- CLIP preprocessing no longer decodes originals at full resolution:
decode_for_thumbnail is generalized into decode_image_scaled with a
cover mode (shortest side >= target) and the embedder decodes JPEGs
at the smallest DCT scale covering its 224px fill-crop, in parallel
via rayon. The forward pass, not image decode, is now the dominant
embedding cost.
- EXIF orientation is now applied before embedding, so rotated photos
embed the way they are displayed (previously embedded sideways).
Existing stored embeddings are unaffected.
Major speedups for large-folder thumbnail generation:
- JPEGs decode through mozjpeg at the smallest DCT scale covering
320px instead of full resolution, with catch_unwind and a fallback
to the image crate for anything mozjpeg rejects. EXIF orientation
preserved in both paths.
- RGB8 pipeline end to end (no RGBA round-trip) and CatmullRom resize
in place of Lanczos3.
- Video posters grab the seeked frame directly instead of running the
ffmpeg thumbnail filter (which decoded ~100 frames), with decode
threads capped at 2 per process.
- Workers only sleep when the queue is empty rather than 250ms after
every batch. Image batches commit before videos start, and videos
run sequentially off the rayon pool (blocking ffmpeg waits were
starving image decode) with per-item commits so progress keeps
moving through slow stretches.
Delete thumbnails alongside DB rows in all three removal paths (watcher
detect, delete-from-disk command, remove-folder command) so orphans no
longer accumulate passively.
Intercept RenameMode::Both watcher events and handle them as in-place
path updates: renames the thumbnail file to the new hash and updates the
DB row without touching the embedding, avoiding a full rebuild on move.
Falls back to thumbnail regeneration if the file rename fails.
db.rs:
- Move idx_images_taken_at creation to after ensure_column so it never
runs against a schema where taken_at doesn't exist yet; this was causing
a startup panic on any DB that predates Phase 1
- COALESCE(taken_at, created_at) → COALESCE(taken_at, modified_at) since
created_at is never populated (modified_at is always set)
- Remove dead is_tagging_job_cancelled function (superseded by
is_tagging_job_processing) and unused thumbnail_path from ImagePathRecord
indexer.rs:
- Watcher create path now calls update_folder_count after commit_batch and
emits folder-counts-changed so the sidebar count stays in sync; the
notification is only emitted when the DB write actually succeeds
store.ts / Timeline.tsx:
- compareImages taken_asc/taken_desc: fall back to modified_at not
created_at (created_at is always null)
- Timeline groupImages: same fallback fix so images group by month
correctly for libraries not yet re-indexed for EXIF
- subscribeToProgress: listen for folder-counts-changed and call
loadFolders() to keep the sidebar image count live
Monitors all registered folders using OS-native events (ReadDirectoryChangesW
on Windows) so new, modified, and deleted files are reflected in the gallery
automatically without manual reindexing.
Key design points:
- Adaptive blocking: recv() when no events pending (zero CPU), switches to
recv_timeout(earliest_deadline) only when debounce timers are running — wakes
exactly when the soonest event is ready, no busy-polling
- 500 ms per-path debounce coalesces rapid OS event bursts into one action
- Change detection preserved: build_record skips upsert if file_size + mtime
unchanged, preventing spurious thumbnail/embedding re-queues and avoiding
clobbering of existing metadata like thumbnail_path
- Access events (reads) filtered out; only Create/Modify/Remove trigger work
- Deletion path: emits watcher-deleted event with image IDs; frontend removes
those images from state and clears selectedImage if it was deleted
WatcherHandle stored in app state; add_folder / remove_folder / update_folder_path
commands keep the watched path set in sync with the DB.
Adds EXIF date extraction during indexing so photos can be sorted by when
they were actually taken, not just when the file was modified on disk.
- Add `taken_at` (nullable TEXT, ISO 8601) column to the images table via
ensure_column migration and a new idx_images_taken_at index
- Populate taken_at in build_record via extract_exif_date, which tries
DateTimeOriginal → DateTimeDigitized → DateTime tags and rejects all-zero
sentinel dates written by uninitialised cameras
- upsert_image uses taken_at = excluded.taken_at so a re-indexed file gets
fresh EXIF; stale dates from replaced files are not preserved
- Add taken_asc / taken_desc sort using COALESCE(taken_at, created_at) so
images without EXIF gracefully fall back to their creation date
- Surface as "Taken: newest / oldest" in the Toolbar sort dropdown
- Shift map_image_row column indices after the new taken_at (index 10)
Adds a full discovery and organisation layer to the gallery.
## AI Tagger
- WD tagger (ONNX via ort) with DirectML/CPU acceleration
- Per-image and per-folder tag queuing; configurable batch size and
confidence threshold; model downloaded on first use via ureq/zip
- Tags stored with source ('ai' | 'user'); user tags are never
overwritten by AI; AI tags protected from accidental promotion
- Tagger pause/resume per folder; in-flight batches discarded cleanly
on pause or cancellation without leaving jobs stuck in processing
## Semantic & Tag Search
- CLIP text-query embedding via candle (HuggingFace hub model)
- Progressive candidate doubling with filter post-processing so
folder/rating/media-kind filters do not silently under-return results
- Tag search with DB-backed pagination (offset + COUNT total)
- Filename search unchanged; prefix syntax: s:, t:, f: in search bar
## Similarity Search
- HNSW in-memory index (hnsw_rs) with monotonic embedding_revision
counter for cache invalidation; build_index retries if a concurrent
write advances the revision during construction
- Folder-scoped similar search uses brute-force cosine scan (no k limit)
- Region-based similarity: crop an arbitrary rectangle in the lightbox,
embed it with CLIP, find the nearest images globally or within folder
- Pagination for both similar and region results; scope toggle
(all media / current folder) re-runs the query without reopening image
## Duplicate Finder
- Three-phase detection: stat (size) → sample hash (4×16 KB windows)
→ full-file hash (xxh3, large files only) to eliminate false positives
- Filesystem-first deletion: only removes DB rows for files successfully
deleted from disk; failed files remain visible for retry
- Persisted scan cache (SQLite) with invalidation on reindex and deletion;
both global and per-folder scopes invalidated after any deletion
## Tag Cloud & Explore
- Visual cluster explore: k-means over CLIP embeddings, configurable k,
representative thumbnail per cluster; cache keyed on xxh3 of embedding set
- Tag explore: ranked tag list with image counts and representative
thumbnail per tag; invalidated when AI tagging completes or folder removed
- Tag autocomplete for search bar
## Folder Management
- Inline rename (display name only, not OS rename)
- Relocate: file-picker to update folder path; all image paths rewritten
using SUBSTR prefix replacement to avoid corrupting paths that contain
the folder name as a substring
- Missing-folder recovery banner: Locate or Remove, never silent deletion
- Right-click context menu on sidebar items: Reindex, Rename, Locate,
Remove; hover buttons preserved alongside context menu
## Background Tasks
- Per-folder progress panel with embedding, tagging, caption, and
thumbnail stage breakdown
- Retry button per task for failed embeddings and tagging jobs
- Completed-task notifications (system tray via tauri-plugin-notification)
- Scan errors shown inline; WalkDir permission errors protect existing
records from deletion rather than cascading data loss
## Backend correctness
- sqlite-vec DML performed outside transactions throughout (virtual-table
limitation); embedding revision incremented on delete as well as insert
- Tagging job discard checks status = 'processing' so paused jobs reset
to 'pending' are also skipped, not just cancelled ones
- Stale async responses in Lightbox guarded with cancellation flags and
currentImageIdRef for both tag-load and tag-add callbacks
- Duplicate cache cleared on reindex; folder-removal clears vector rows
outside transaction before deleting the folder row
- Fix root cause: embedding_source_path() returns Result<PathBuf>, returning
Err for videos without a thumbnail instead of silently falling back to the
raw .mp4 path that CLIP cannot decode
- indexer: split embedding batch into pre-failed (no source) and embeddable
jobs; pre-failed are marked immediately without hitting the CLIP model
- db: retry_failed_embedding_jobs skips videos still without a thumbnail so
they no longer re-fail immediately on retry
- Add get_failed_embedding_images command listing failed files + error per folder
- Gallery: amber warning badge on tiles with embedding_status = 'failed'
- BackgroundTasks: fetch and show failed filenames/errors in expanded panel
- Toolbar: conditional 'Failed Embeddings' amber filter pill shown when any
folder has embedding_failed > 0; filters at DB level via new
embedding_failed_only param on get_images / count_images
- TagCloud: replaced vocabulary/dictionary label system with representative
image thumbnails per cluster; results cached in SQLite by image-id hash
- add semantic text search with toolbar mode switching and sqlite-vec query support
- improve embedding progress visibility, failure recovery, and similar-image affordances
- add search clearing and keyboard controls for filename vs semantic search modes
- refine background task interactions and gallery/lightbox embedding states
Refs: #4
- add Candle + HF Hub CLIP image embedding pipeline with background embedding worker
- write image embeddings into sqlite-vec and expose similar-image lookup through a new backend command
- surface embedding progress and recovery in the UI, including retries for failed embeddings
- improve gallery/lightbox embedding UX and make similar-image actions directly accessible
Refs: #3, #4
Sets up Tauri v2 + React frontend with SQLite metadata store,
sqlite-vec for CLIP embedding infrastructure, and r2d2 connection
pool replacing the single Arc<Mutex<Connection>>. Indexer now uses
rayon for parallel file metadata collection.
LyAhn