mirror of
https://github.com/kikootwo/ReadMeABook.git
synced 2026-06-03 21:00:09 +00:00
kikootwo
c8bfcdb611
Introduce a Bulk Import feature for admins to scan server folders, match discovered audiobook folders against Audible, review matches, and queue batch imports. What changed: - Added documentation: documentation/features/bulk-import.md and TABLEOFCONTENTS update. - Backend: SSE scan endpoint (POST /api/admin/bulk-import/scan) streams discovery and matching events; execute endpoint (POST /api/admin/bulk-import/execute) validates paths, creates/resolves audiobook & request records, and queues organize_files jobs. Both endpoints enforce admin-only access and validate allowed root directories (download_dir, media_dir, /bookdrop). - Frontend: Modal wizard and steps for folder selection, scan progress, and match review (BulkImportWizard + ScanFolderStep, ScanProgressStep, MatchReviewStep + shared types). - Utilities: bulk-import-scanner for folder discovery and ffprobe metadata extraction; shared types for scanned books/events. - UI: Added Bulk Import quick action to admin dashboard (src/app/admin/page.tsx). Key details: - Audible searches are rate-limited (≈1.5s) and matching results include library/request status checks. - Reuses existing organize_files job queue and manual-import pipeline; no new database tables introduced (state is ephemeral during the wizard). - Includes error handling, path normalization, and security checks for allowed directories. This commit wires frontend, backend, and docs together to provide an admin-only multi-step bulk import workflow.
83 lines
4.2 KiB
Markdown
83 lines
4.2 KiB
Markdown
# Bulk Import Feature
|
|
|
|
**Status:** ✅ Implemented | Admin-only | Multi-step wizard modal
|
|
|
|
## Overview
|
|
Lets admins scan a server folder recursively, discover audiobook subfolders, match against Audible, review matches, and import selected books via the existing manual import pipeline.
|
|
|
|
## Flow
|
|
1. **Select Folder** — Browse base folders (Downloads, Media Library, Book Drop), pick scan root
|
|
2. **Scan & Match** — Recursively discover audiobook folders (max 10 levels), read metadata via ffprobe, search Audible per book (1.5s rate limit)
|
|
3. **Review & Import** — Scrollable list with skip toggles, library status, confidence badges; Start Import queues organize_files jobs
|
|
|
|
## Key Details
|
|
- **Access:** Admin-only, modal opened from admin dashboard Quick Actions
|
|
- **Audio detection:** Uses `AUDIO_EXTENSIONS` from `src/lib/constants/audio-formats.ts`
|
|
- **Audiobook boundary:** A folder containing audio files = one audiobook; subfolders not scanned further
|
|
- **Metadata extraction:** ffprobe reads `album` (title), `album_artist` (author), `composer` (narrator) from first audio file
|
|
- **Fallback:** If metadata tags are empty, folder name used as search term; "Low Confidence" badge shown
|
|
- **Author/narrator dedup:** Splits on `,;& ` delimiters, removes names appearing in both fields
|
|
- **Scan depth:** Max 10 levels recursion
|
|
- **Rate limiting:** 1.5s delay between Audible searches (same as existing scraping rate limit)
|
|
- **Library check:** Uses `findPlexMatch()` for ASIN-based availability detection
|
|
- **Import:** Reuses existing `organize_files` job queue (same as manual import)
|
|
- **No new database tables** — all state is ephemeral during wizard session
|
|
|
|
## API Endpoints
|
|
|
|
**POST /api/admin/bulk-import/scan** (SSE stream)
|
|
- Body: `{ rootPath: string }`
|
|
- Path validation: must be within download_dir, media_dir, or /bookdrop
|
|
- Streams events: `progress`, `discovery_complete`, `matching`, `book_matched`, `complete`, `error`
|
|
- Each `book_matched` event includes: folderPath, match (Audible data), inLibrary, hasActiveRequest, metadataSource
|
|
|
|
**POST /api/admin/bulk-import/execute**
|
|
- Body: `{ imports: Array<{ folderPath: string, asin: string }> }`
|
|
- Creates audiobook records + requests, queues organize_files jobs
|
|
- Returns: `{ success, results[], summary: { total, succeeded, failed } }`
|
|
|
|
## SSE Event Types
|
|
|
|
| Event | Data | When |
|
|
|---|---|---|
|
|
| `progress` | `{ phase, foldersScanned, audiobooksFound, currentFolder }` | During folder discovery |
|
|
| `discovery_complete` | `{ totalFound, message }` | All folders scanned |
|
|
| `matching` | `{ current, total, folderName, searchTerm }` | Before each Audible search |
|
|
| `book_matched` | Full book result with match data | After each Audible search |
|
|
| `complete` | `{ audiobooks[], totalFound, matched, inLibrary }` | All matching done |
|
|
| `error` | `{ message }` | On failure |
|
|
|
|
## UI States
|
|
|
|
| State | Visual |
|
|
|---|---|
|
|
| Normal (will import) | Full opacity, blue toggle ON |
|
|
| Skipped by user | 40% opacity, gray toggle OFF |
|
|
| Already in library | 40% opacity, green "In Library" badge, toggle disabled |
|
|
| Active request exists | 40% opacity, purple "Requested" badge, toggle disabled |
|
|
| No Audible match | Red "No Match" badge, folder name shown, pre-skipped |
|
|
| Low confidence (folder name fallback) | Amber "Low Confidence" badge |
|
|
|
|
## Files
|
|
|
|
**Backend:**
|
|
- `src/lib/utils/bulk-import-scanner.ts` — Folder discovery + ffprobe metadata
|
|
- `src/app/api/admin/bulk-import/scan/route.ts` — SSE scan endpoint
|
|
- `src/app/api/admin/bulk-import/execute/route.ts` — Batch import endpoint
|
|
|
|
**Frontend:**
|
|
- `src/components/admin/BulkImportWizard.tsx` — Modal orchestrator
|
|
- `src/components/admin/bulk-import/types.ts` — Shared types
|
|
- `src/components/admin/bulk-import/ScanFolderStep.tsx` — Folder browser
|
|
- `src/components/admin/bulk-import/ScanProgressStep.tsx` — Progress display
|
|
- `src/components/admin/bulk-import/MatchReviewStep.tsx` — Review list + import
|
|
|
|
**Modified:**
|
|
- `src/app/admin/page.tsx` — Added Bulk Import quick action + modal
|
|
|
|
## Related
|
|
- [Manual Import](manual-import.md) — Single-book import (reused pipeline)
|
|
- [File Organization](../phase3/file-organization.md) — organize_files job
|
|
- [Audible Integration](../integrations/audible.md) — Search/scraping
|
|
- [Background Jobs](../backend/services/jobs.md) — Job queue system
|