Add admin Bulk Import feature

Introduce a Bulk Import feature for admins to scan server folders, match discovered audiobook folders against Audible, review matches, and queue batch imports.

What changed:
- Added documentation: documentation/features/bulk-import.md and TABLEOFCONTENTS update.
- Backend: SSE scan endpoint (POST /api/admin/bulk-import/scan) streams discovery and matching events; execute endpoint (POST /api/admin/bulk-import/execute) validates paths, creates/resolves audiobook & request records, and queues organize_files jobs. Both endpoints enforce admin-only access and validate allowed root directories (download_dir, media_dir, /bookdrop).
- Frontend: Modal wizard and steps for folder selection, scan progress, and match review (BulkImportWizard + ScanFolderStep, ScanProgressStep, MatchReviewStep + shared types).
- Utilities: bulk-import-scanner for folder discovery and ffprobe metadata extraction; shared types for scanned books/events.
- UI: Added Bulk Import quick action to admin dashboard (src/app/admin/page.tsx).

Key details:
- Audible searches are rate-limited (≈1.5s) and matching results include library/request status checks.
- Reuses existing organize_files job queue and manual-import pipeline; no new database tables introduced (state is ephemeral during the wizard).
- Includes error handling, path normalization, and security checks for allowed directories.

This commit wires frontend, backend, and docs together to provide an admin-only multi-step bulk import workflow.
This commit is contained in:
kikootwo
2026-03-13 12:03:21 -04:00
parent 6fc622c4e7
commit c8bfcdb611
11 changed files with 2275 additions and 1 deletions
+4
View File
@@ -99,6 +99,7 @@
## Admin Features
- **Dashboard (metrics, downloads, requests)** → [admin-dashboard.md](admin-dashboard.md)
- **Bulk import (scan folders, match Audible, batch import)** → [features/bulk-import.md](features/bulk-import.md)
- **Jobs management UI** → [backend/services/scheduler.md](backend/services/scheduler.md)
- **Request deletion (soft delete, seeding awareness)** → [admin-features/request-deletion.md](admin-features/request-deletion.md)
- **Request approval system, auto-approve settings** → [admin-features/request-approval.md](admin-features/request-approval.md)
@@ -167,3 +168,6 @@
**"How do Hardcover shelves work?"** → [backend/services/hardcover-sync.md](backend/services/hardcover-sync.md)
**"How do I add a new shelf provider?"** → [backend/services/goodreads-sync.md](backend/services/goodreads-sync.md#adding-a-new-provider)
**"How does the shelf sync core work?"** → [backend/services/goodreads-sync.md](backend/services/goodreads-sync.md#shared-sync-core)
**"How does bulk import work?"** → [features/bulk-import.md](features/bulk-import.md)
**"How do I import multiple audiobooks at once?"** → [features/bulk-import.md](features/bulk-import.md)
**"How does the bulk import scanner detect audiobooks?"** → [features/bulk-import.md](features/bulk-import.md)
+82
View File
@@ -0,0 +1,82 @@
# Bulk Import Feature
**Status:** ✅ Implemented | Admin-only | Multi-step wizard modal
## Overview
Lets admins scan a server folder recursively, discover audiobook subfolders, match against Audible, review matches, and import selected books via the existing manual import pipeline.
## Flow
1. **Select Folder** — Browse base folders (Downloads, Media Library, Book Drop), pick scan root
2. **Scan & Match** — Recursively discover audiobook folders (max 10 levels), read metadata via ffprobe, search Audible per book (1.5s rate limit)
3. **Review & Import** — Scrollable list with skip toggles, library status, confidence badges; Start Import queues organize_files jobs
## Key Details
- **Access:** Admin-only, modal opened from admin dashboard Quick Actions
- **Audio detection:** Uses `AUDIO_EXTENSIONS` from `src/lib/constants/audio-formats.ts`
- **Audiobook boundary:** A folder containing audio files = one audiobook; subfolders not scanned further
- **Metadata extraction:** ffprobe reads `album` (title), `album_artist` (author), `composer` (narrator) from first audio file
- **Fallback:** If metadata tags are empty, folder name used as search term; "Low Confidence" badge shown
- **Author/narrator dedup:** Splits on `,;& ` delimiters, removes names appearing in both fields
- **Scan depth:** Max 10 levels recursion
- **Rate limiting:** 1.5s delay between Audible searches (same as existing scraping rate limit)
- **Library check:** Uses `findPlexMatch()` for ASIN-based availability detection
- **Import:** Reuses existing `organize_files` job queue (same as manual import)
- **No new database tables** — all state is ephemeral during wizard session
## API Endpoints
**POST /api/admin/bulk-import/scan** (SSE stream)
- Body: `{ rootPath: string }`
- Path validation: must be within download_dir, media_dir, or /bookdrop
- Streams events: `progress`, `discovery_complete`, `matching`, `book_matched`, `complete`, `error`
- Each `book_matched` event includes: folderPath, match (Audible data), inLibrary, hasActiveRequest, metadataSource
**POST /api/admin/bulk-import/execute**
- Body: `{ imports: Array<{ folderPath: string, asin: string }> }`
- Creates audiobook records + requests, queues organize_files jobs
- Returns: `{ success, results[], summary: { total, succeeded, failed } }`
## SSE Event Types
| Event | Data | When |
|---|---|---|
| `progress` | `{ phase, foldersScanned, audiobooksFound, currentFolder }` | During folder discovery |
| `discovery_complete` | `{ totalFound, message }` | All folders scanned |
| `matching` | `{ current, total, folderName, searchTerm }` | Before each Audible search |
| `book_matched` | Full book result with match data | After each Audible search |
| `complete` | `{ audiobooks[], totalFound, matched, inLibrary }` | All matching done |
| `error` | `{ message }` | On failure |
## UI States
| State | Visual |
|---|---|
| Normal (will import) | Full opacity, blue toggle ON |
| Skipped by user | 40% opacity, gray toggle OFF |
| Already in library | 40% opacity, green "In Library" badge, toggle disabled |
| Active request exists | 40% opacity, purple "Requested" badge, toggle disabled |
| No Audible match | Red "No Match" badge, folder name shown, pre-skipped |
| Low confidence (folder name fallback) | Amber "Low Confidence" badge |
## Files
**Backend:**
- `src/lib/utils/bulk-import-scanner.ts` — Folder discovery + ffprobe metadata
- `src/app/api/admin/bulk-import/scan/route.ts` — SSE scan endpoint
- `src/app/api/admin/bulk-import/execute/route.ts` — Batch import endpoint
**Frontend:**
- `src/components/admin/BulkImportWizard.tsx` — Modal orchestrator
- `src/components/admin/bulk-import/types.ts` — Shared types
- `src/components/admin/bulk-import/ScanFolderStep.tsx` — Folder browser
- `src/components/admin/bulk-import/ScanProgressStep.tsx` — Progress display
- `src/components/admin/bulk-import/MatchReviewStep.tsx` — Review list + import
**Modified:**
- `src/app/admin/page.tsx` — Added Bulk Import quick action + modal
## Related
- [Manual Import](manual-import.md) — Single-book import (reused pipeline)
- [File Organization](../phase3/file-organization.md) — organize_files job
- [Audible Integration](../integrations/audible.md) — Search/scraping
- [Background Jobs](../backend/services/jobs.md) — Job queue system