JezzWTF/phokus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: expand media discovery and AI workflows #8

Merged
LyAhn merged 25 commits from codex/5-discovery-features into main 2026-06-07 22:43:16 +00:00
Conversation 0 Commits 25 Files Changed 32
+9539 -760
LyAhn commented 2026-06-06 19:10:24 +00:00
Owner
Copy Link
Copy Source

Overview

Merges codex/5-discovery-features into main — 11 commits, 32 files changed (+8740 / -694). Expands media discovery with region-based similarity search, adds an AI tagging pipeline (WD SwinV2 tagger), hardens indexing reliability, and polishes the overall UX.

Changes

Region-Based Similarity Search

  • New "Search within image" button in the lightbox — draw a crop region to find visually similar images using that crop's embedding
  • Crops embedded in-memory via a new embed_image_crop method on ClipImageEmbedder (no temp file)
  • Adds folder-scoped cosine search (search_image_ids_by_embedding_in_folder) and a new find_similar_by_region Tauri command, wired end-to-end through the Zustand store and Lightbox UI

WD AI Tagger

  • New tagger.rs implementing WD SwinV2 tagger v3 with DirectML/CPU ONNX acceleration
  • CSV-backed tag labels; model downloaded on demand via ureq/zip
  • Per-image tag queuing from the lightbox sidebar; tags refresh live after inference completes
  • AI Tags button hidden for video files; new Tauri commands for model status, download, queue, and tag CRUD

AI Captioning (Backend Only)

  • Caption worker introduced (captioner.rs) and then intentionally disabled in lib.rs — backend code intact for future re-enabling
  • All caption UI surfaces removed from Lightbox, BackgroundTasks, and SettingsModal

Media Exploration & Tag Discovery

  • Richer exploration and tagging controls in the gallery UI
  • Tags used by a single image now appear in Explore (previously only multi-image tags shown)
  • Tag cache invalidated after edits so changes are reflected immediately

Indexing Reliability

  • sqlite-vec embedding deletions moved outside the image transaction to prevent transactional virtual-table failures
  • Terminal index-progress event emitted on indexing errors so the frontend reloads partial state and clears its active scan indicator

UX & Safety

  • Time-limited confirmation step required before removing an indexed folder from the sidebar
  • Desktop notifications on completion for folder scans, embeddings, AI tagging, and duplicate scans (including failure counts)

Branding & Docs

  • App title set to Phokus; default Vite favicon removed
  • README rewritten to reflect current feature set; CLAUDE.md added

Testing Notes

  • WD tagger requires a one-time model download via Settings; DirectML accelerates on compatible GPUs, falls back to CPU
  • Region search requires images to have embeddings generated first
  • Caption worker is disabled — no captioning UI or workflows are active in this build
## Overview Merges `codex/5-discovery-features` into `main` — 11 commits, 32 files changed (+8740 / -694). Expands media discovery with region-based similarity search, adds an AI tagging pipeline (WD SwinV2 tagger), hardens indexing reliability, and polishes the overall UX. ## Changes ### Region-Based Similarity Search - New "Search within image" button in the lightbox — draw a crop region to find visually similar images using that crop's embedding - Crops embedded in-memory via a new `embed_image_crop` method on `ClipImageEmbedder` (no temp file) - Adds folder-scoped cosine search (`search_image_ids_by_embedding_in_folder`) and a new `find_similar_by_region` Tauri command, wired end-to-end through the Zustand store and Lightbox UI ### WD AI Tagger - New `tagger.rs` implementing WD SwinV2 tagger v3 with DirectML/CPU ONNX acceleration - CSV-backed tag labels; model downloaded on demand via `ureq`/`zip` - Per-image tag queuing from the lightbox sidebar; tags refresh live after inference completes - AI Tags button hidden for video files; new Tauri commands for model status, download, queue, and tag CRUD ### AI Captioning (Backend Only) - Caption worker introduced (`captioner.rs`) and then intentionally disabled in `lib.rs` — backend code intact for future re-enabling - All caption UI surfaces removed from Lightbox, BackgroundTasks, and SettingsModal ### Media Exploration & Tag Discovery - Richer exploration and tagging controls in the gallery UI - Tags used by a single image now appear in Explore (previously only multi-image tags shown) - Tag cache invalidated after edits so changes are reflected immediately ### Indexing Reliability - `sqlite-vec` embedding deletions moved outside the image transaction to prevent transactional virtual-table failures - Terminal `index-progress` event emitted on indexing errors so the frontend reloads partial state and clears its active scan indicator ### UX & Safety - Time-limited confirmation step required before removing an indexed folder from the sidebar - Desktop notifications on completion for folder scans, embeddings, AI tagging, and duplicate scans (including failure counts) ### Branding & Docs - App title set to **Phokus**; default Vite favicon removed - README rewritten to reflect current feature set; `CLAUDE.md` added ## Testing Notes - WD tagger requires a one-time model download via Settings; DirectML accelerates on compatible GPUs, falls back to CPU - Region search requires images to have embeddings generated first - Caption worker is disabled — no captioning UI or workflows are active in this build
LyAhn added 11 commits 2026-06-06 19:10:24 +00:00
feat: add local AI captions and queue controls bee6adc61a
feat: add WD tagger, remove caption UI, add per-image tag queuing in lightbox a9dd2b2797 
- Add WD SwinV2 tagger v3 backend (tagger.rs) with DirectML acceleration
- Expose tagger commands (model status, download, queue, tags CRUD)
- Add queueTaggingForImage store action and AI tags button in lightbox sidebar
- Remove all caption UI surfaces (Lightbox, BackgroundTasks, SettingsModal)
- Disable caption worker in lib.rs; keep Rust backend intact for future use
- Make lightbox sidebar scrollable with sticky header/footer
- Collapse long tag lists (show 8, expandable) in lightbox sidebar
fix: refresh lightbox tags live after AI tagging, hide AI tags button for video f93a80bc87
feat: add WD tagger with CSV tag support and model download via ureq/zip b2826d1143 
Introduces tagger.rs for WD-based image tagging with CSV label support.
Adds csv, ureq, and zip dependencies; updates gitignore to exclude Python/JSON files.
feat: expand media exploration and tagging controls ff4a568b57
feat: add region-based similarity search df17497808 
Adds a "Search within image" button to the lightbox that lets the user
draw a crop region on an image and find visually similar results using
that crop's embedding. The crop is embedded in-memory without a temp
file via a new embed_image_crop method on ClipImageEmbedder.

Introduces a folder-scoped cosine search (search_image_ids_by_embedding_in_folder)
to support the current-folder scope option, and wires up the new
find_similar_by_region Tauri command end-to-end from Rust through to
the Zustand store and Lightbox UI.
chore: set app title to Phokus, remove default Vite favicon 5b35bc5b6e
docs: rewrite README to reflect current features, add CLAUDE.md d7595703de
fix(indexer): recover cleanly from indexing failures 6824dcffb3 
Move sqlite-vec embedding deletions outside the image transaction to avoid transactional virtual-table failures. Emit a terminal indexing progress event on errors so the frontend reloads partial state and clears its active scan state.
feat(discovery): surface all tags and guard folder removal 2cdab000fb 
Include tags used by a single image in Explore and invalidate the tag cache after edits so changes appear immediately. Add an accessible, time-limited confirmation step before removing indexed folders from the sidebar.
feat(notifications): report completed background tasks c1070649fa 
Register the Tauri notification plugin and request notification permission during startup. Send completion notifications for folder scans, embeddings, AI tagging, and duplicate scans, including failure counts where available.
LyAhn added 1 commit 2026-06-06 20:51:48 +00:00
fix: address four correctness bugs in duplicate detection and tagging f82002129a 
- Duplicate scanner now runs a full-file hash pass (Phase 3) for large
  files (> 64 KB) after the sample-hash shortlist, preventing false
  positives from reaching the destructive deletion workflow
- delete_images_from_disk now attempts filesystem removal before touching
  the DB; only successfully deleted files have their rows removed,
  keeping locked/permission-denied files visible in the library
- AI tag upsert guards user-sourced rows with WHERE source != 'user' so
  a tagger re-run cannot silently overwrite manually added tags
- clear_tagging_jobs excludes 'cancelled' rows from the immediate DELETE
  so the worker can observe cancellation via is_tagging_job_cancelled
  before the rows are cleaned up at next startup
LyAhn added 1 commit 2026-06-06 21:02:19 +00:00
fix: four more correctness bugs from PR review de0c2ab12d 
- Duplicate scanner now emits one DuplicateGroup per distinct full hash
  within a sample-hash bucket, preventing disjoint sets (A,A vs B,B)
  from being merged and presented as a single deletable group
- Tagger model download now installs the shared ONNX runtime DLLs via
  ensure_onnx_runtime so the tagger is ready on a clean install without
  requiring the caption model to have been downloaded first
- Caption queue clear now follows the tagging worker pattern: marks
  in-flight rows as cancelled and deletes only non-processing rows;
  the caption worker skips writing results for cancelled jobs; startup
  cleanup removes any leftover cancelled caption rows
- Region search pagination now stores the crop rect in state and uses
  find_similar_by_region for subsequent pages instead of falling through
  to loadImages which appended unrelated folder results
LyAhn added 1 commit 2026-06-06 21:24:23 +00:00
fix: three more correctness bugs from PR review 9e08d9cce3 
- upsert_image now resets ai_tagged_at, ai_rating, ai_tagger_model, and
  ai_tagger_error to NULL and deletes AI-sourced image_tags when a file
  is reindexed after content changes, preventing stale tags/ratings from
  the previous version being shown and blocking re-tagging
- get_pending_tagging_jobs_excluding now applies the paused-folder
  exclusion in SQL before LIMIT using folder_exclusion_clause, matching
  the pattern used by caption and embedding job claimers; previously a
  large paused-folder backlog could starve all unpaused folders
- Merged similarRequestToken into galleryRequestToken so all
  gallery-producing requests share one generation counter; a late
  similarity response can no longer overwrite a subsequently loaded
  folder, and vice versa
LyAhn added 2 commits 2026-06-07 08:17:20 +00:00
fix: five PR review bugs and folder-recovery UX for missing/renamed folders 7871d52d39 
BackgroundTasks: dismiss no longer calls clearTaggingJobs; dismissal only
updates local dismissed state

DuplicateFinder: entering the view clears stale groups when the stored scan
scope differs from selectedFolderId; duplicateScanFolderId is recorded on
each scan and cache load so scope is always tracked

Tagger threshold: set_tagger_threshold now sets TAGGER_SESSION_DIRTY so the
worker rebuilds its WdTagger instance and applies the new threshold on the
next batch

Region search global scope: fetch offset+limit+2 candidates before removing
the source image so has_more is accurate across pages

Region search pagination: loadMoreImages now sets loadingImages and checks
galleryRequestToken before appending results, preventing duplicate pages and
stale responses corrupting a new collection

Folder recovery: when index_folder detects a missing path it records the
error in a new folders.scan_error column (ensure_column migration) and emits
done without deleting images, preserving them for recovery. A new
update_folder_path command rewrites image paths in the DB before reindexing
so thumbnails and embeddings are not needlessly regenerated for unchanged
files. The sidebar shows an amber warning icon on affected folders and a
recovery banner with Locate Folder and Remove actions
feat(folders): add rename, missing-folder recovery, and right-click context menu 4e5923ba84 
- Backend: new `rename_folder` command (updates display name only, not path)
  and `update_folder_path` command (rewrites image paths in DB before reindexing
  so thumbnails/embeddings are not regenerated unnecessarily)
- DB: `rename_folder` and `update_folder_path` helpers; `scan_error` column on
  folders table to surface indexing failures without silently dropping images
- Store: `renameFolder` and `updateFolderPath` actions
- Sidebar: right-click context menu on each folder row (Reindex, Rename, Locate
  Folder, Remove from app); inline rename input on Enter/Escape/blur; missing-
  folder recovery banner with Locate/Remove; hover icon buttons (reindex + remove
  with Confirm/Cancel) preserved alongside the context menu
LyAhn added 1 commit 2026-06-07 08:46:20 +00:00
fix: address all 6 P2 review issues from PR #8 round 4 183cdff6b1 
- BackgroundTasks: per-task Retry now dispatches to the correct backend —
  retryFailedEmbeddings for embedding failures, queueTaggingJobs for tagging
  failures, or both; previously always called embedding retry only

- commands: tag-cloud cache key now hashes both image IDs and embedding bytes
  (xxh3) so re-embedding a file after reindex correctly invalidates the cache;
  removed now-dead fnv_hash_ids helper

- db: update_folder_path uses SUBSTR instead of replace() to rewrite only the
  leading path prefix, preventing corruption when the old folder name also
  appears in a subdirectory name

- db: add_user_tag promotes an existing AI-owned tag to user-owned on conflict
  instead of DO NOTHING, preventing the tagger from deleting a manually added
  tag that coincides with an AI tag

- db/commands: add clear_duplicate_scan_cache / invalidate_duplicate_scan_cache
  so deletion removes the stale persisted cache entry; without this a restart
  would reload groups containing deleted images

- store: setSimilarScope re-runs loadSimilarByRegion (with saved crop) when the
  active collection is Region Search Results, instead of silently switching to
  whole-image similarity and discarding the crop
LyAhn added 1 commit 2026-06-07 09:30:42 +00:00
fix: address 3 P2 review issues from PR #8 round 5 04ade3d5d6 
captioner: set_caption_detail now marks CAPTION_SESSION_DIRTY so the
cached FlorenceCaptioner session is torn down and rebuilt with the new
prompt and token limit; previously only the file was updated on disk

db: reset_generated_captions now cancels in-flight caption jobs instead
of deleting them; the worker checks for a cancelled row before writing
back, so deleting the row allowed results to be written immediately after
a reset — now mirrors the clear_caption_jobs cancellation protocol

BackgroundTasks: caption_pending/ready/failed counts are now included in
pendingMediaWork, task stages, hasFailed checks, and the dismiss snapshot;
previously a folder with only caption work caused the panel to disappear
entirely and exposed no caption progress or failure status
LyAhn added 1 commit 2026-06-07 21:17:46 +00:00
fix: address 4 P2 review issues from PR #8 round 6 7efb187971 
db/commands: paginate tag search — add offset + COUNT query to
search_images_by_tag returning (images, total); TagSearchParams gains
offset field and command returns TagSearchPage; store handles both the
initial load and load-more path so results beyond PAGE_SIZE are reachable

commands: semantic search over-fetches by 4× when any post-query filter
(folder, media kind, favorites, rating) is active, then truncates to the
requested limit; previously well-rated matches ranked just beyond the bare
vector limit were silently omitted

db: wrap update_folder_path in a transaction so a path-rewrite failure
(e.g. uniqueness collision) rolls back the already-updated folder row
instead of leaving the DB in an inconsistent half-migrated state

store: invalidate exploreTagsFolderId when a tagging run completes so
Explore reflects the updated tag distribution without requiring a manual
folder switch; previously the cache was never busted by background tagging
LyAhn added 1 commit 2026-06-07 21:30:20 +00:00
fix: address P1 and P2 review issues from PR #8 round 7 20c43d2056 
indexer: clear duplicate scan cache (folder + global) at the end of
every do_index so stale groups are never presented after a reindex;
previously a file modified or replaced on disk would still appear as a
duplicate candidate from the cached result until a manual rescan

db/vector: replace timestamp-based HNSW revision with a monotonically
incremented integer counter (app_kv.embedding_revision); mark_embedding_ready
atomically increments the counter so two embeddings saved within the same
second correctly advance the revision; get_embedding_revision now reads
this counter instead of MAX(embedding_updated_at), preventing the HNSW
cache from serving stale vectors after same-second batch updates
LyAhn added 1 commit 2026-06-07 21:55:16 +00:00
fix(search): close three P2 correctness gaps from code review ae3d6e1034 
- vector: increment embedding_revision whenever delete_embedding is
  called so the HNSW cache is invalidated after image/folder deletions,
  not just after inserts

- hnsw_index: fix race in build_index — read the revision before
  fetching embeddings and again after the parallel_insert; retry the
  whole build if the revision advanced during construction, ensuring
  the cached index always reflects a consistent snapshot

- commands: replace the fixed 4× over-fetch in semantic_search_images
  with a progressive doubling loop; start at exactly `limit` candidates
  and double the batch (capped at 8192) until the page is filled or the
  vector table is exhausted, preventing both under- and over-fetching
LyAhn added 1 commit 2026-06-07 22:03:15 +00:00
fix(duplicates): close three more P2 correctness issues from review 1f6650021c 
- commands: delete_images_from_disk now returns Vec<i64> of IDs that
  were actually removed from disk rather than a bare count, so callers
  can act precisely on what succeeded vs. what failed

- store: deleteSelectedDuplicates uses the returned ID set to filter
  duplicate groups (files that failed to delete stay visible for retry)
  and invalidates both the global "all" cache and each affected folder
  cache, not just the currently-viewed scope

- Lightbox: guard the getImageTags effect with a cancellation flag so
  a response that arrives after the user has navigated to another image
  cannot overwrite the new image's tag list
LyAhn added 1 commit 2026-06-07 22:10:51 +00:00
fix: pre-empt review findings before next push 7ead26d7fb 
- hnsw_index: fetch folder embedding IDs before acquiring the read lock
  in find_similar_image_matches; holding the lock across a SQLite query
  was delaying concurrent ensure_index write-lock requests

- store/DuplicateFinder: add duplicateScanError state; scanDuplicates
  now catches errors and stores them rather than silently leaving the
  user with empty results and no feedback
LyAhn added 1 commit 2026-06-07 22:25:42 +00:00
fix: four review issues (search cap, tagging pause, stale tag, walkdir) ba989b37b9 
- commands: semantic search doubling loop now returns when batch reaches
  the 8192 cap rather than re-fetching the same IDs forever when filters
  yield fewer than limit matches (P1 infinite loop)

- db/indexer: tagging worker checks is_tagging_job_processing instead
  of is_tagging_job_cancelled so paused jobs reset to 'pending' are also
  discarded; cancelled rows are deleted, paused rows kept for retry

- Lightbox: capture image id at form-submit time and compare against
  currentImageIdRef in the addUserTag callback, matching the guard
  already applied to the getImageTags effect

- indexer: WalkDir errors are no longer silently swallowed; paths under
  unreadable subtrees are excluded from the missing-file deletion pass
  using Path::starts_with (component-aware) so a transient permission
  error cannot cascade into data loss or affect sibling directories
LyAhn added 1 commit 2026-06-07 22:35:31 +00:00
fix: three pre-emptive issues caught before push fd1585b5e2 
- store: removeFolder now also resets exploreTagsFolderId so the Explore
  tag list is invalidated when a folder is removed, not just tagCloud

- db: delete_folder moves vector::delete_embedding calls outside the
  transaction — sqlite-vec virtual-table DML is unreliable inside a
  transaction and caused remove_folder to fail for folders with embeddings

- store: loadMoreImages now passes similarSourceFolderId as the fourth
  arg to loadSimilarImages; omitting it caused each paginated load to
  overwrite the state with similarFolderId, corrupting scope-switches
  that use similarSourceFolderId afterward
LyAhn merged commit 0ca4d142d8 into main 2026-06-07 22:43:16 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
bug
Something isn't working
 documentation
Improvements or additions to documentation
 duplicate
This issue or pull request already exists
 enhancement
New feature or request
 good first issue
Good for newcomers
 help wanted
Extra attention is needed
 invalid
This doesn't seem right
 question
Further information is requested
 wontfix
This will not be worked on
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
LyAhn
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: JezzWTF/phokus#8
Delete Branch "codex/5-discovery-features"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?