mirror of
https://github.com/kikootwo/ReadMeABook.git
synced 2026-06-02 20:30:10 +00:00
Add first-class ebook request support and UI
Implements first-class ebook requests with their own type, parent-child relationship to audiobook requests, and separate status flow. Updates database schema and migrations to support 'type' and 'parentRequestId' fields on requests. Adds processors and job types for ebook search and direct HTTP download from Anna's Archive, with FlareSolverr integration for Cloudflare bypass. Enhances admin UI tables and request actions to display and manage ebook requests, including orange badge and source links. Updates documentation to reflect new ebook support, configuration, and behavior.
This commit is contained in:
kikootwo
@@ -2,6 +2,8 @@
|
||||
|
||||
**Critical:** This document defines AI-optimized documentation standards and development workflow. **NEVER PERFORM COMMITS ON THE REPOSITORY.**
|
||||
|
||||
**ALWAYS DO:** When you feel work is complete, use the docker compose build readmebook to confirm you have no errors. If the build succeeds, then you can tell me it is ready to be tested.
|
||||
|
||||
---
|
||||
|
||||
## 1. Token-Efficient Documentation System
|
||||
|
||||
@@ -38,10 +38,12 @@
|
||||
- **Database caching, real-time matching** → [integrations/audible.md](integrations/audible.md)
|
||||
- **Book covers API for login page** → [frontend/pages/login.md](frontend/pages/login.md)
|
||||
|
||||
## E-book Sidecar
|
||||
- **Optional e-book downloads from Anna's Archive** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
## E-book Support (First-Class)
|
||||
- **First-class ebook requests, separate tracking** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
- **ASIN-based matching, format selection** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
- **Non-blocking, atomic failures** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
- **Ebook ranking algorithm (inverted size scoring)** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
- **Direct HTTP downloads from Anna's Archive** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
- **Ebook delete behavior (files only)** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
|
||||
## Automation Pipeline
|
||||
- **Full pipeline overview** → [phase3/README.md](phase3/README.md)
|
||||
@@ -108,8 +110,10 @@
|
||||
**"How do Usenet/NZB downloads work?"** → [phase3/sabnzbd.md](phase3/sabnzbd.md), [backend/services/jobs.md](backend/services/jobs.md)
|
||||
**"Can I use both qBittorrent and SABnzbd?"** → [phase3/download-clients.md](phase3/download-clients.md)
|
||||
**"How does Plex matching work?"** → [integrations/plex.md](integrations/plex.md)
|
||||
**"How does e-book sidecar work?"** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
**"How does e-book support work?"** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
**"How do I enable e-book downloads?"** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md), [settings-pages.md](settings-pages.md)
|
||||
**"What happens when I delete an ebook request?"** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md#delete-behavior)
|
||||
**"Why do ebook requests have an orange badge?"** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md#ui-representation)
|
||||
**"How do scheduled jobs work?"** → [backend/services/scheduler.md](backend/services/scheduler.md)
|
||||
**"How do I configure external services?"** → [setup-wizard.md](setup-wizard.md), [settings-pages.md](settings-pages.md)
|
||||
**"What's the database schema?"** → [backend/database.md](backend/database.md)
|
||||
|
||||
@@ -1,19 +1,29 @@
|
||||
# E-book Sidecar
|
||||
# E-book Support
|
||||
|
||||
**Status:** ✅ Implemented | Optional e-book downloads from Anna's Archive
|
||||
**Status:** ✅ Implemented | First-class ebook requests with Anna's Archive integration
|
||||
|
||||
## Overview
|
||||
Automatically downloads e-books from Anna's Archive to accompany audiobooks, placing them in the same folder.
|
||||
Ebooks are first-class citizens in RMAB, with their own request type, tracking, and UI representation. When an audiobook request completes, an ebook request is automatically created (if enabled). Ebooks are downloaded directly from Anna's Archive via HTTP.
|
||||
|
||||
## Key Details
|
||||
- **When:** Runs during file organization (after audiobook copied, after cover art)
|
||||
- **Matching:** ASIN-based search (exact match)
|
||||
- **Non-blocking:** Failures don't affect audiobook download
|
||||
- **Atomic:** Either succeeds or fails gracefully
|
||||
- **Location:** E-book placed in same directory as audiobook
|
||||
- **Filename:** `[Title] - [Author].[format]` (sanitized)
|
||||
|
||||
## Configuration
|
||||
### First-Class Ebook Requests
|
||||
- **Request Type:** `type: 'ebook'` (vs `'audiobook'`)
|
||||
- **Parent Relationship:** Ebook requests are children of audiobook requests (`parentRequestId`)
|
||||
- **Terminal State:** `downloaded` (ebooks don't have "available" state like audiobooks)
|
||||
- **UI Badge:** Orange (#f16f19) ebook badge to distinguish from audiobooks
|
||||
- **Separate Tracking:** Own progress, status, and error handling
|
||||
|
||||
### Flow
|
||||
1. Audiobook organization completes
|
||||
2. Ebook request created automatically (if enabled)
|
||||
3. `search_ebook` job searches Anna's Archive
|
||||
4. `start_direct_download` downloads via HTTP
|
||||
5. `organize_files` copies to audiobook folder
|
||||
6. Request marked as `downloaded` (terminal)
|
||||
7. "Available" notification sent
|
||||
|
||||
### Configuration
|
||||
|
||||
**Admin Settings → E-book Sidecar tab**
|
||||
|
||||
@@ -21,287 +31,155 @@ Automatically downloads e-books from Anna's Archive to accompany audiobooks, pla
|
||||
|-----|---------|---------|-------------|
|
||||
| `ebook_sidecar_enabled` | `false` | `true/false` | Enable feature |
|
||||
| `ebook_sidecar_preferred_format` | `epub` | `epub, pdf, mobi, azw3, any` | Preferred format |
|
||||
| `ebook_sidecar_base_url` | `https://annas-archive.li` | URL | Base URL (mirror resilience) |
|
||||
| `ebook_sidecar_flaresolverr_url` | `` (empty) | URL | FlareSolverr proxy URL (optional) |
|
||||
| `ebook_sidecar_base_url` | `https://annas-archive.li` | URL | Base URL |
|
||||
| `ebook_sidecar_flaresolverr_url` | `` (empty) | URL | FlareSolverr proxy (optional) |
|
||||
|
||||
**Stored in:** `Configuration` table (database)
|
||||
## Database Schema
|
||||
|
||||
**Request model additions:**
|
||||
```prisma
|
||||
type String @default("audiobook") // 'audiobook' | 'ebook'
|
||||
parentRequestId String? @map("parent_request_id")
|
||||
parentRequest Request? @relation("EbookParent", fields: [parentRequestId], references: [id])
|
||||
childRequests Request[] @relation("EbookParent")
|
||||
```
|
||||
|
||||
**Indexes:** `type`, `parentRequestId`
|
||||
|
||||
## Job Processors
|
||||
|
||||
### search_ebook
|
||||
- Searches Anna's Archive by ASIN first, then title + author
|
||||
- Creates download history record with `downloadClient: 'direct'`
|
||||
- Triggers `start_direct_download` job
|
||||
|
||||
### start_direct_download
|
||||
- Downloads file via HTTP with progress tracking
|
||||
- Tries multiple slow download links on failure
|
||||
- Triggers `organize_files` on success
|
||||
|
||||
### monitor_direct_download
|
||||
- Future use for async download monitoring
|
||||
- Currently, most tracking happens in start_direct_download
|
||||
|
||||
## Ranking Algorithm
|
||||
|
||||
Ebook ranking (for future multi-source support):
|
||||
- **Format Score:** 40 pts (exact match) to 10 pts (different format)
|
||||
- **Size Score:** 30 pts (inverse - smaller files preferred)
|
||||
- **Source Score:** 30 pts (Anna's Archive gets full score)
|
||||
|
||||
## Delete Behavior
|
||||
|
||||
**Ebook deletion is different from audiobook deletion:**
|
||||
- Only deletes ebook files (`.epub`, `.pdf`, `.mobi`, etc.)
|
||||
- Does NOT delete the title folder (audiobook files remain)
|
||||
- Does NOT delete from backend library (Plex/ABS)
|
||||
- Does NOT clear audiobook availability linkage
|
||||
- Soft-deletes the ebook request record
|
||||
|
||||
## UI Representation
|
||||
|
||||
### RequestCard
|
||||
- Orange ebook badge displayed next to status badge
|
||||
- Orange book icon for placeholder cover art
|
||||
- Interactive search disabled (Anna's Archive only)
|
||||
|
||||
### Status Flow
|
||||
```
|
||||
pending → searching → downloading → processing → downloaded (terminal)
|
||||
↘ awaiting_search (retry) ↗
|
||||
```
|
||||
|
||||
## FlareSolverr Integration
|
||||
|
||||
Anna's Archive uses Cloudflare protection which may block direct scraping requests. FlareSolverr solves this by using a headless browser to bypass the protection.
|
||||
|
||||
### What is FlareSolverr?
|
||||
- Proxy server using headless Chrome/Chromium
|
||||
- Automatically solves Cloudflare challenges
|
||||
- Returns HTML content after challenge is solved
|
||||
- Open source: https://github.com/FlareSolverr/FlareSolverr
|
||||
|
||||
### When to Use FlareSolverr
|
||||
- **Required:** When e-book downloads consistently fail with no search results
|
||||
- **Optional:** If direct requests work (depends on Cloudflare's current state)
|
||||
- **Recommended:** For reliable, consistent downloads
|
||||
Anna's Archive uses Cloudflare protection. FlareSolverr bypasses this using a headless browser.
|
||||
|
||||
### Setup
|
||||
1. Run FlareSolverr via Docker:
|
||||
```bash
|
||||
docker run -d --name flaresolverr -p 8191:8191 ghcr.io/flaresolverr/flaresolverr:latest
|
||||
```
|
||||
2. In Admin Settings → E-book Sidecar, enter: `http://localhost:8191`
|
||||
3. Click "Test Connection" to verify
|
||||
```bash
|
||||
docker run -d --name flaresolverr -p 8191:8191 ghcr.io/flaresolverr/flaresolverr:latest
|
||||
```
|
||||
|
||||
### How It Works
|
||||
1. Requests are routed through FlareSolverr
|
||||
2. FlareSolverr loads the page in headless Chrome
|
||||
3. If Cloudflare challenge appears, it waits for solution
|
||||
4. HTML is returned after page loads
|
||||
5. Falls back to direct requests if FlareSolverr fails
|
||||
Configure URL in Admin Settings → E-book Sidecar: `http://localhost:8191`
|
||||
|
||||
### Performance Impact
|
||||
- **First request:** ~5-10 seconds (browser startup)
|
||||
- **Subsequent requests:** ~2-5 seconds per page
|
||||
- **Total time:** ~15-30 seconds per e-book (vs ~5-15 without)
|
||||
### Performance
|
||||
- First request: ~5-10 seconds
|
||||
- Subsequent: ~2-5 seconds per page
|
||||
- Total: ~15-30 seconds per ebook
|
||||
|
||||
## How It Works
|
||||
## Scraping Strategy
|
||||
|
||||
### Flow
|
||||
1. **Trigger:** File organization completes audiobook copy
|
||||
2. **Check:** `ebook_sidecar_enabled === 'true'`
|
||||
3. **Search:** Try ASIN first (if available), then fall back to title + author
|
||||
4. **Extract MD5:** First search result → MD5 hash
|
||||
5. **Get Download Links:** Find "no waitlist" slow download links
|
||||
6. **Extract URL:** Parse slow download page for actual file server URL
|
||||
7. **Download:** Stream file to audiobook directory
|
||||
8. **Rename:** Sanitize filename based on metadata
|
||||
|
||||
### Scraping Strategy
|
||||
|
||||
**Method 1: ASIN Search (exact match)**
|
||||
### Method 1: ASIN Search (exact match)
|
||||
```
|
||||
Search: https://annas-archive.li/search?ext=epub&lang=en&q="asin:B09TWSRMCB"
|
||||
↓
|
||||
MD5 Page: https://annas-archive.li/md5/[md5]
|
||||
↓ (Filter: "slow partner server" links)
|
||||
Slow Download: https://annas-archive.li/slow_download/[md5]/0/5
|
||||
↓ (Parse for actual download URL)
|
||||
File Server: http://[server-ip]:port/path/to/file.epub
|
||||
```
|
||||
|
||||
**Method 2: Title + Author Search (fallback)**
|
||||
```
|
||||
Search: https://annas-archive.li/search?q=The+Housemaid+Freida+McFadden
|
||||
&ext=epub
|
||||
&content=book_nonfiction&content=book_fiction&content=book_unknown
|
||||
&lang=en
|
||||
↓
|
||||
(Same flow as ASIN search from MD5 page onwards)
|
||||
Slow Download: https://annas-archive.li/slow_download/[md5]/0/5
|
||||
↓
|
||||
File Server: http://[server]/path/to/file.epub
|
||||
```
|
||||
|
||||
### Matching Priority
|
||||
1. **ASIN** (exact match - most accurate, if available)
|
||||
2. **Title + Author** (fuzzy match with book/language filters)
|
||||
|
||||
### Retry Logic
|
||||
- **Max attempts:** 5 slow download links
|
||||
- **Timeout:** 60 seconds per download
|
||||
- **Delays:** 1.5 seconds between requests
|
||||
- **Retries:** 3x for 5xx errors with exponential backoff
|
||||
|
||||
## Format Support
|
||||
|
||||
| Format | Extension | Recommended | Notes |
|
||||
|--------|-----------|-------------|-------|
|
||||
| EPUB | `.epub` | ✅ Yes | Most compatible with e-readers |
|
||||
| PDF | `.pdf` | ⚠️ Sometimes | Best for fixed-layout books |
|
||||
| MOBI | `.mobi` | ⚠️ Legacy | Kindle (older devices) |
|
||||
| AZW3 | `.azw3` | ⚠️ Sometimes | Kindle (newer devices) |
|
||||
| Any | `[first available]` | ❌ No | Downloads first match |
|
||||
|
||||
**Recommendation:** Use EPUB for maximum compatibility.
|
||||
### Method 2: Title + Author (fallback)
|
||||
```
|
||||
Search: https://annas-archive.li/search?q=Title+Author&ext=epub&lang=en
|
||||
↓ (Same flow from MD5 page)
|
||||
```
|
||||
|
||||
## File Naming
|
||||
|
||||
**Pattern:** `[Title] - [Author].[format]`
|
||||
|
||||
**Sanitization:**
|
||||
- Remove invalid chars: `<>:"/\|?*`
|
||||
- Collapse multiple spaces
|
||||
- Trim leading/trailing spaces and dots
|
||||
- Limit to 100 characters
|
||||
|
||||
**Examples:**
|
||||
- `The Housemaid - Freida McFadden.epub`
|
||||
- `Project Hail Mary - Andy Weir.pdf`
|
||||
- Remove: `<>:"/\|?*`
|
||||
- Collapse spaces, trim, limit to 200 chars
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Graceful Failures (non-blocking):**
|
||||
- No ASIN available → Skip silently (log info)
|
||||
- No search results → Log warning, continue audiobook
|
||||
- No download links → Log warning, continue audiobook
|
||||
- All downloads fail → Log error, continue audiobook
|
||||
- Download timeout → Log error, continue audiobook
|
||||
**Non-blocking errors:**
|
||||
- No search results → Request goes to `awaiting_search` for retry
|
||||
- All downloads fail → Same retry behavior
|
||||
- Audiobook organization never affected
|
||||
|
||||
**Never Blocks Audiobook:**
|
||||
- All e-book errors are non-fatal
|
||||
- Audiobook organization completes regardless
|
||||
- Errors logged to job events (visible in admin)
|
||||
## Technical Files
|
||||
|
||||
## Logging
|
||||
**Processors:**
|
||||
- `src/lib/processors/search-ebook.processor.ts`
|
||||
- `src/lib/processors/direct-download.processor.ts`
|
||||
- `src/lib/processors/organize-files.processor.ts` (ebook branch)
|
||||
|
||||
**Success (with FlareSolverr):**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Using FlareSolverr at http://localhost:8191
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
Found via ASIN: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found MD5: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found 8 download link(s)
|
||||
Attempting download link 1/5...
|
||||
Downloading from: 93.123.118.12
|
||||
E-book downloaded: The Housemaid - Freida McFadden.epub
|
||||
```
|
||||
**Services:**
|
||||
- `src/lib/services/ebook-scraper.ts`
|
||||
- `src/lib/services/job-queue.service.ts` (ebook job types)
|
||||
|
||||
**Success (ASIN match, direct):**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
Found via ASIN: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found MD5: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found 8 download link(s)
|
||||
Attempting download link 1/5...
|
||||
Downloading from: 93.123.118.12
|
||||
E-book downloaded: The Housemaid - Freida McFadden.epub
|
||||
```
|
||||
**Utils:**
|
||||
- `src/lib/utils/file-organizer.ts` (`organizeEbook` method)
|
||||
- `src/lib/utils/ranking-algorithm.ts` (`rankEbooks` function)
|
||||
|
||||
**Success (Title fallback):**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
No results for ASIN, falling back to title + author search...
|
||||
Searching by title + author: "The Housemaid" by Freida McFadden...
|
||||
Found via title search: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found MD5: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found 8 download link(s)
|
||||
E-book downloaded: The Housemaid - Freida McFadden.epub
|
||||
```
|
||||
**UI:**
|
||||
- `src/components/requests/RequestCard.tsx` (ebook badge)
|
||||
|
||||
**Failure:**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
No results for ASIN, falling back to title + author search...
|
||||
Searching by title + author: "The Housemaid" by Freida McFadden...
|
||||
No search results found for title: "The Housemaid" by Freida McFadden
|
||||
E-book download failed: No search results found (tried ASIN and title+author)
|
||||
```
|
||||
**Delete:**
|
||||
- `src/lib/services/request-delete.service.ts` (ebook-specific logic)
|
||||
|
||||
## Troubleshooting
|
||||
## Format Support
|
||||
|
||||
### E-book Not Downloaded
|
||||
|
||||
**Cause:** No matching e-book in Anna's Archive (tried ASIN and title+author)
|
||||
**Solution:** Not all audiobooks have e-book equivalents, this is expected
|
||||
|
||||
**Cause:** ASIN mismatch (Anna's Archive has different ASIN)
|
||||
**Solution:** Feature now automatically falls back to title + author search
|
||||
|
||||
**Cause:** All download links failed
|
||||
**Solution:** Check job logs for errors, may be temporary server issues
|
||||
|
||||
### Wrong Format Downloaded
|
||||
|
||||
**Cause:** Preferred format not available
|
||||
**Solution:** Anna's Archive doesn't have that format, falls back to available format
|
||||
|
||||
### Download Timeout
|
||||
|
||||
**Cause:** Slow file server or large file
|
||||
**Solution:** Automatic retry with next download link
|
||||
|
||||
### Feature Not Working
|
||||
|
||||
**Cause:** Feature disabled
|
||||
**Solution:** Admin Settings → E-book Sidecar → Enable toggle
|
||||
|
||||
### Cloudflare Blocking
|
||||
|
||||
**Cause:** Anna's Archive has Cloudflare protection enabled
|
||||
**Solution:** Configure FlareSolverr (see FlareSolverr Integration section)
|
||||
|
||||
**Symptoms:**
|
||||
- No search results found
|
||||
- Requests timing out
|
||||
- Errors about Cloudflare challenge
|
||||
|
||||
### FlareSolverr Not Working
|
||||
|
||||
**Cause:** FlareSolverr not running or unreachable
|
||||
**Solution:**
|
||||
1. Verify FlareSolverr is running: `docker ps | grep flaresolverr`
|
||||
2. Check URL is correct (usually `http://localhost:8191`)
|
||||
3. Test connection in Admin Settings
|
||||
|
||||
**Cause:** FlareSolverr timing out
|
||||
**Solution:** FlareSolverr may need more time; check container logs for errors
|
||||
|
||||
## Security & Legal
|
||||
|
||||
**Important Notes:**
|
||||
- Anna's Archive is a shadow library
|
||||
- Use at your own discretion and responsibility
|
||||
- Ensure compliance with local laws and regulations
|
||||
- Feature is optional and disabled by default
|
||||
- No API key required (web scraping)
|
||||
|
||||
**Privacy:**
|
||||
- User-Agent: `ReadMeABook/1.0 (Audiobook Automation)`
|
||||
- No tracking or analytics
|
||||
- Distributed (each user scrapes for themselves)
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
**Files:**
|
||||
- Service: `src/lib/services/ebook-scraper.ts`
|
||||
- Integration: `src/lib/utils/file-organizer.ts` (line 265+)
|
||||
- Settings API: `src/app/api/admin/settings/ebook/route.ts`
|
||||
- FlareSolverr Test API: `src/app/api/admin/settings/ebook/test-flaresolverr/route.ts`
|
||||
- UI: `src/app/admin/settings/page.tsx` (ebook tab)
|
||||
|
||||
**Dependencies:**
|
||||
- axios (HTTP requests)
|
||||
- cheerio (HTML parsing)
|
||||
- fs/promises (file operations)
|
||||
|
||||
**Caching:**
|
||||
- MD5 lookups cached in-memory (prevents re-scraping same ASIN)
|
||||
- Cache cleared on service restart
|
||||
|
||||
## Performance
|
||||
|
||||
**Impact:**
|
||||
- **Network:** 3-5 requests per e-book (search, MD5, slow download pages)
|
||||
- **Time:** ~5-15 seconds per e-book (depends on file server)
|
||||
- **Storage:** E-books typically 1-50 MB
|
||||
- **CPU:** Minimal (streaming download)
|
||||
| Format | Extension | Recommended |
|
||||
|--------|-----------|-------------|
|
||||
| EPUB | `.epub` | ✅ Yes |
|
||||
| PDF | `.pdf` | ⚠️ Sometimes |
|
||||
| MOBI | `.mobi` | ⚠️ Legacy |
|
||||
| AZW3 | `.azw3` | ⚠️ Sometimes |
|
||||
|
||||
## Limitations
|
||||
|
||||
1. **Match Accuracy:** Title + author search may return wrong book if title is common
|
||||
2. **Format Availability:** Depends on Anna's Archive catalog
|
||||
3. **Download Speed:** Depends on file server load
|
||||
4. **Language:** Title search filters for English books only
|
||||
5. **Success Rate:** ~70-90% (ASIN has higher accuracy, title fallback is less precise)
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- ISBN-13 fallback matching (between ASIN and title search)
|
||||
- Format preference priority list (try EPUB, then PDF, then MOBI)
|
||||
- Per-request override (API endpoint)
|
||||
- Statistics tracking (success rate, formats, match method)
|
||||
- Rate limit monitoring
|
||||
- Relevance scoring for title search results
|
||||
1. Single source (Anna's Archive) - future Prowlarr support stubbed
|
||||
2. Title search may return wrong book for common titles
|
||||
3. Download speed depends on file server load
|
||||
4. English books only (title search filter)
|
||||
|
||||
## Related
|
||||
- [File Organization](../phase3/file-organization.md) - Where e-book download happens
|
||||
- [File Organization](../phase3/file-organization.md) - Ebook organization
|
||||
- [Settings Pages](../settings-pages.md) - Configuration UI
|
||||
- [Configuration Service](../backend/services/config.md) - Settings storage
|
||||
- [Ranking Algorithm](../phase3/ranking-algorithm.md) - Ebook ranking
|
||||
- [Request Deletion](../admin-features/request-deletion.md) - Delete behavior
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE "requests" ADD COLUMN IF NOT EXISTS "type" TEXT NOT NULL DEFAULT 'audiobook';
|
||||
ALTER TABLE "requests" ADD COLUMN IF NOT EXISTS "parent_request_id" TEXT;
|
||||
|
||||
-- CreateIndex
|
||||
CREATE INDEX IF NOT EXISTS "requests_type_idx" ON "requests"("type");
|
||||
CREATE INDEX IF NOT EXISTS "requests_parent_request_id_idx" ON "requests"("parent_request_id");
|
||||
|
||||
-- AddForeignKey (with ON DELETE SET NULL)
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM pg_constraint
|
||||
WHERE conname = 'requests_parent_request_id_fkey'
|
||||
) THEN
|
||||
ALTER TABLE "requests" ADD CONSTRAINT "requests_parent_request_id_fkey"
|
||||
FOREIGN KEY ("parent_request_id") REFERENCES "requests"("id") ON DELETE SET NULL ON UPDATE CASCADE;
|
||||
END IF;
|
||||
END $$;
|
||||
+13
-2
@@ -212,7 +212,8 @@ model Request {
|
||||
audiobookId String @map("audiobook_id")
|
||||
status String @default("pending")
|
||||
// Status values: pending, awaiting_approval, denied, searching, downloading, processing, downloaded, available, failed, cancelled, awaiting_search, awaiting_import, warn
|
||||
// Flow: pending → searching → downloading → processing → downloaded → available (when matched in Plex)
|
||||
// Flow (audiobook): pending → searching → downloading → processing → downloaded → available (when matched in Plex)
|
||||
// Flow (ebook): pending → searching → downloading → processing → downloaded (terminal - no available state)
|
||||
progress Int @default(0) // 0-100
|
||||
priority Int @default(0)
|
||||
errorMessage String? @map("error_message") @db.Text
|
||||
@@ -227,6 +228,11 @@ model Request {
|
||||
updatedAt DateTime @updatedAt @map("updated_at")
|
||||
completedAt DateTime? @map("completed_at")
|
||||
|
||||
// Request type: 'audiobook' (default) or 'ebook'
|
||||
// Ebook requests are created automatically when an audiobook is organized (if ebook downloads enabled)
|
||||
type String @default("audiobook") // 'audiobook' | 'ebook'
|
||||
parentRequestId String? @map("parent_request_id") // Links ebook request to originating audiobook request
|
||||
|
||||
// Soft delete support
|
||||
deletedAt DateTime? @map("deleted_at")
|
||||
deletedBy String? @map("deleted_by") // Admin user ID
|
||||
@@ -236,12 +242,16 @@ model Request {
|
||||
audiobook Audiobook @relation(fields: [audiobookId], references: [id], onDelete: Cascade)
|
||||
downloadHistory DownloadHistory[]
|
||||
jobs Job[]
|
||||
parentRequest Request? @relation("EbookParent", fields: [parentRequestId], references: [id], onDelete: SetNull)
|
||||
childRequests Request[] @relation("EbookParent")
|
||||
|
||||
@@index([userId])
|
||||
@@index([audiobookId])
|
||||
@@index([status])
|
||||
@@index([createdAt(sort: Desc)])
|
||||
@@index([deletedAt])
|
||||
@@index([type])
|
||||
@@index([parentRequestId])
|
||||
@@map("requests")
|
||||
}
|
||||
|
||||
@@ -260,7 +270,7 @@ model DownloadHistory {
|
||||
leechers Int?
|
||||
qualityScore Int? @map("quality_score")
|
||||
selected Boolean @default(false)
|
||||
downloadClient String? @map("download_client") // qbittorrent, sabnzbd
|
||||
downloadClient String? @map("download_client") // qbittorrent, sabnzbd, direct (HTTP download for ebooks)
|
||||
downloadClientId String? @map("download_client_id")
|
||||
downloadStatus String? @map("download_status")
|
||||
// Status values: queued, downloading, completed, failed, stalled
|
||||
@@ -302,6 +312,7 @@ model Job {
|
||||
requestId String? @map("request_id")
|
||||
type String
|
||||
// Job types: search_indexers, monitor_download, organize_files, scan_plex, plex_recently_added_check, match_plex
|
||||
// Ebook job types: search_ebook, start_direct_download, monitor_direct_download
|
||||
status String @default("pending")
|
||||
// Status values: pending, active, completed, failed, delayed, stuck
|
||||
priority Int @default(0)
|
||||
|
||||
@@ -16,6 +16,7 @@ interface ActiveDownload {
|
||||
eta: number | null;
|
||||
user: string;
|
||||
startedAt: Date;
|
||||
type?: 'audiobook' | 'ebook';
|
||||
}
|
||||
|
||||
interface ActiveDownloadsTableProps {
|
||||
@@ -77,7 +78,7 @@ export function ActiveDownloadsTable({ downloads }: ActiveDownloadsTableProps) {
|
||||
<thead className="bg-gray-50 dark:bg-gray-900 border-b border-gray-200 dark:border-gray-700">
|
||||
<tr>
|
||||
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
|
||||
Audiobook
|
||||
Request
|
||||
</th>
|
||||
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
|
||||
User
|
||||
@@ -104,8 +105,21 @@ export function ActiveDownloadsTable({ downloads }: ActiveDownloadsTableProps) {
|
||||
>
|
||||
<td className="px-6 py-4">
|
||||
<div>
|
||||
<div className="text-sm font-medium text-gray-900 dark:text-gray-100">
|
||||
{download.title}
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-sm font-medium text-gray-900 dark:text-gray-100">
|
||||
{download.title}
|
||||
</span>
|
||||
{download.type === 'ebook' && (
|
||||
<span
|
||||
className="inline-flex items-center gap-1 px-2 py-0.5 text-xs font-medium rounded-full"
|
||||
style={{ backgroundColor: '#f16f1920', color: '#f16f19' }}
|
||||
>
|
||||
<svg className="w-3 h-3" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path d="M9 4.804A7.968 7.968 0 005.5 4c-1.255 0-2.443.29-3.5.804v10A7.969 7.969 0 015.5 14c1.669 0 3.218.51 4.5 1.385A7.962 7.962 0 0114.5 14c1.255 0 2.443.29 3.5.804v-10A7.968 7.968 0 0014.5 4c-1.255 0-2.443.29-3.5.804V12a1 1 0 11-2 0V4.804z" />
|
||||
</svg>
|
||||
Ebook
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<div className="text-sm text-gray-500 dark:text-gray-400">
|
||||
{download.author}
|
||||
|
||||
@@ -18,6 +18,7 @@ interface RecentRequest {
|
||||
title: string;
|
||||
author: string;
|
||||
status: string;
|
||||
type?: 'audiobook' | 'ebook';
|
||||
user: string;
|
||||
createdAt: Date;
|
||||
completedAt: Date | null;
|
||||
@@ -237,7 +238,7 @@ export function RecentRequestsTable({ requests, ebookSidecarEnabled = false }: R
|
||||
<thead className="bg-gray-50 dark:bg-gray-900 border-b border-gray-200 dark:border-gray-700">
|
||||
<tr>
|
||||
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
|
||||
Audiobook
|
||||
Request
|
||||
</th>
|
||||
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wider">
|
||||
User
|
||||
@@ -264,8 +265,21 @@ export function RecentRequestsTable({ requests, ebookSidecarEnabled = false }: R
|
||||
>
|
||||
<td className="px-6 py-4">
|
||||
<div>
|
||||
<div className="text-sm font-medium text-gray-900 dark:text-gray-100">
|
||||
{request.title}
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-sm font-medium text-gray-900 dark:text-gray-100">
|
||||
{request.title}
|
||||
</span>
|
||||
{request.type === 'ebook' && (
|
||||
<span
|
||||
className="inline-flex items-center gap-1 px-2 py-0.5 text-xs font-medium rounded-full"
|
||||
style={{ backgroundColor: '#f16f1920', color: '#f16f19' }}
|
||||
>
|
||||
<svg className="w-3 h-3" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path d="M9 4.804A7.968 7.968 0 005.5 4c-1.255 0-2.443.29-3.5.804v10A7.969 7.969 0 015.5 14c1.669 0 3.218.51 4.5 1.385A7.962 7.962 0 0114.5 14c1.255 0 2.443.29 3.5.804v-10A7.968 7.968 0 0014.5 4c-1.255 0-2.443.29-3.5.804V12a1 1 0 11-2 0V4.804z" />
|
||||
</svg>
|
||||
Ebook
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<div className="text-sm text-gray-500 dark:text-gray-400">
|
||||
{request.author}
|
||||
@@ -280,7 +294,9 @@ export function RecentRequestsTable({ requests, ebookSidecarEnabled = false }: R
|
||||
<td className="px-6 py-4 text-sm text-gray-900 dark:text-gray-100">
|
||||
{request.user}
|
||||
</td>
|
||||
<td className="px-6 py-4">{getStatusBadge(request.status)}</td>
|
||||
<td className="px-6 py-4">
|
||||
{getStatusBadge(request.status)}
|
||||
</td>
|
||||
<td className="px-6 py-4 text-sm text-gray-500 dark:text-gray-400">
|
||||
{formatDistanceToNow(new Date(request.createdAt), { addSuffix: true })}
|
||||
</td>
|
||||
@@ -298,6 +314,7 @@ export function RecentRequestsTable({ requests, ebookSidecarEnabled = false }: R
|
||||
title: request.title,
|
||||
author: request.author,
|
||||
status: request.status,
|
||||
type: request.type,
|
||||
torrentUrl: request.torrentUrl,
|
||||
}}
|
||||
onDelete={handleDeleteClick}
|
||||
|
||||
@@ -18,6 +18,7 @@ export interface RequestActionsDropdownProps {
|
||||
title: string;
|
||||
author: string;
|
||||
status: string;
|
||||
type?: 'audiobook' | 'ebook';
|
||||
torrentUrl?: string | null;
|
||||
};
|
||||
onDelete: (requestId: string, title: string) => void;
|
||||
@@ -41,15 +42,41 @@ export function RequestActionsDropdown({
|
||||
const [showInteractiveSearch, setShowInteractiveSearch] = useState(false);
|
||||
const { containerRef, dropdownRef, positionAbove, style } = useSmartDropdownPosition(isOpen);
|
||||
|
||||
// Determine available actions based on status
|
||||
const canSearch = ['pending', 'failed', 'awaiting_search'].includes(request.status);
|
||||
// Determine request type
|
||||
const isEbook = request.type === 'ebook';
|
||||
|
||||
// Determine available actions based on status and type
|
||||
// Ebooks don't support manual/interactive search (Anna's Archive only)
|
||||
const canSearch = !isEbook && ['pending', 'failed', 'awaiting_search'].includes(request.status);
|
||||
const canCancel = ['pending', 'searching', 'downloading'].includes(request.status);
|
||||
const canDelete = true; // Admins can always delete
|
||||
// Only show "View Source" if we have a valid indexer page URL (not a magnet link)
|
||||
const canViewSource = !!request.torrentUrl &&
|
||||
!request.torrentUrl.startsWith('magnet:') &&
|
||||
|
||||
// View Source: For ebooks, extract MD5 from slow download URL and link to Anna's Archive
|
||||
// For audiobooks, show indexer page URL (not magnet links)
|
||||
let viewSourceUrl: string | null = null;
|
||||
if (isEbook && request.torrentUrl) {
|
||||
// torrentUrl for ebooks is JSON array of slow download URLs
|
||||
// Extract MD5 from URL pattern: /slow_download/[md5]/...
|
||||
try {
|
||||
const urls = JSON.parse(request.torrentUrl);
|
||||
if (Array.isArray(urls) && urls.length > 0) {
|
||||
const md5Match = urls[0].match(/\/slow_download\/([a-f0-9]{32})\//i);
|
||||
if (md5Match) {
|
||||
viewSourceUrl = `https://annas-archive.li/md5/${md5Match[1]}`;
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Not JSON, ignore
|
||||
}
|
||||
} else if (request.torrentUrl && !request.torrentUrl.startsWith('magnet:')) {
|
||||
viewSourceUrl = request.torrentUrl;
|
||||
}
|
||||
|
||||
const canViewSource = !!viewSourceUrl &&
|
||||
['downloading', 'processing', 'downloaded', 'available'].includes(request.status);
|
||||
const canFetchEbook = ebookSidecarEnabled && ['downloaded', 'available'].includes(request.status);
|
||||
|
||||
// "Try to fetch Ebook" only for audiobook requests
|
||||
const canFetchEbook = !isEbook && ebookSidecarEnabled && ['downloaded', 'available'].includes(request.status);
|
||||
|
||||
// Close dropdown when clicking outside
|
||||
useEffect(() => {
|
||||
@@ -166,9 +193,9 @@ export function RequestActionsDropdown({
|
||||
)}
|
||||
|
||||
{/* View Source */}
|
||||
{canViewSource && (
|
||||
{canViewSource && viewSourceUrl && (
|
||||
<a
|
||||
href={request.torrentUrl!}
|
||||
href={viewSourceUrl}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
onClick={() => setIsOpen(false)}
|
||||
|
||||
@@ -17,7 +17,7 @@ export async function GET(request: NextRequest) {
|
||||
return requireAuth(request, async (req: AuthenticatedRequest) => {
|
||||
return requireAdmin(req, async () => {
|
||||
try {
|
||||
// Get active downloads with related data
|
||||
// Get active downloads with related data (both audiobook and ebook)
|
||||
const activeDownloads = await prisma.request.findMany({
|
||||
where: {
|
||||
status: 'downloading',
|
||||
@@ -26,6 +26,7 @@ export async function GET(request: NextRequest) {
|
||||
select: {
|
||||
id: true,
|
||||
status: true,
|
||||
type: true, // 'audiobook' or 'ebook'
|
||||
progress: true,
|
||||
updatedAt: true,
|
||||
audiobook: {
|
||||
@@ -54,6 +55,8 @@ export async function GET(request: NextRequest) {
|
||||
torrentName: true,
|
||||
torrentHash: true,
|
||||
nzbId: true,
|
||||
downloadClient: true, // qbittorrent, sabnzbd, or direct
|
||||
torrentSizeBytes: true,
|
||||
startedAt: true,
|
||||
createdAt: true,
|
||||
},
|
||||
@@ -75,19 +78,38 @@ export async function GET(request: NextRequest) {
|
||||
let speed = 0;
|
||||
let eta: number | null = null;
|
||||
|
||||
const downloadHistory = download.downloadHistory[0];
|
||||
const downloadClient = downloadHistory?.downloadClient;
|
||||
|
||||
try {
|
||||
if (clientType === 'qbittorrent') {
|
||||
if (downloadClient === 'direct') {
|
||||
// Direct HTTP download (ebooks) - estimate speed from progress and time elapsed
|
||||
const startedAt = downloadHistory?.startedAt || downloadHistory?.createdAt;
|
||||
const totalSize = downloadHistory?.torrentSizeBytes ? Number(downloadHistory.torrentSizeBytes) : 0;
|
||||
|
||||
if (startedAt && download.progress > 0 && totalSize > 0) {
|
||||
const elapsedMs = Date.now() - new Date(startedAt).getTime();
|
||||
const elapsedSeconds = elapsedMs / 1000;
|
||||
const bytesDownloaded = (download.progress / 100) * totalSize;
|
||||
|
||||
if (elapsedSeconds > 0) {
|
||||
speed = Math.round(bytesDownloaded / elapsedSeconds);
|
||||
const remainingBytes = totalSize - bytesDownloaded;
|
||||
eta = speed > 0 ? Math.round(remainingBytes / speed) : null;
|
||||
}
|
||||
}
|
||||
} else if (downloadClient === 'qbittorrent' || (!downloadClient && clientType === 'qbittorrent')) {
|
||||
// Get torrent hash from download history
|
||||
const torrentHash = download.downloadHistory[0]?.torrentHash;
|
||||
const torrentHash = downloadHistory?.torrentHash;
|
||||
if (torrentHash) {
|
||||
const qbService = await getQBittorrentService();
|
||||
const torrentInfo = await qbService.getTorrent(torrentHash);
|
||||
speed = torrentInfo.dlspeed;
|
||||
eta = torrentInfo.eta > 0 ? torrentInfo.eta : null;
|
||||
}
|
||||
} else if (clientType === 'sabnzbd') {
|
||||
} else if (downloadClient === 'sabnzbd' || (!downloadClient && clientType === 'sabnzbd')) {
|
||||
// Get NZB ID from download history
|
||||
const nzbId = download.downloadHistory[0]?.nzbId;
|
||||
const nzbId = downloadHistory?.nzbId;
|
||||
if (nzbId) {
|
||||
const sabnzbdService = await getSABnzbdService();
|
||||
const nzbInfo = await sabnzbdService.getNZB(nzbId);
|
||||
@@ -107,13 +129,14 @@ export async function GET(request: NextRequest) {
|
||||
title: download.audiobook.title,
|
||||
author: download.audiobook.author,
|
||||
status: download.status,
|
||||
type: download.type, // 'audiobook' or 'ebook'
|
||||
progress: download.progress,
|
||||
speed,
|
||||
eta,
|
||||
torrentName: download.downloadHistory[0]?.torrentName || null,
|
||||
downloadStatus: download.downloadHistory[0]?.downloadStatus || null,
|
||||
torrentName: downloadHistory?.torrentName || null,
|
||||
downloadStatus: downloadHistory?.downloadStatus || null,
|
||||
user: download.user.plexUsername,
|
||||
startedAt: download.downloadHistory[0]?.startedAt || download.downloadHistory[0]?.createdAt || download.updatedAt,
|
||||
startedAt: downloadHistory?.startedAt || downloadHistory?.createdAt || download.updatedAt,
|
||||
};
|
||||
})
|
||||
);
|
||||
|
||||
@@ -55,6 +55,7 @@ export async function GET(request: NextRequest) {
|
||||
title: request.audiobook.title,
|
||||
author: request.audiobook.author,
|
||||
status: request.status,
|
||||
type: request.type, // 'audiobook' or 'ebook'
|
||||
user: request.user.plexUsername,
|
||||
createdAt: request.createdAt,
|
||||
completedAt: request.completedAt,
|
||||
|
||||
@@ -61,13 +61,14 @@ export async function POST(request: NextRequest) {
|
||||
const body = await req.json();
|
||||
const { audiobook, torrent } = RequestWithTorrentSchema.parse(body);
|
||||
|
||||
// First check: Is there an existing request in 'downloaded' or 'available' status?
|
||||
// First check: Is there an existing audiobook request in 'downloaded' or 'available' status?
|
||||
// This catches the gap where files are organized but Plex hasn't scanned yet
|
||||
const existingActiveRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
audiobook: {
|
||||
audibleAsin: audiobook.asin,
|
||||
},
|
||||
type: 'audiobook', // Only check audiobook requests (ebook requests are separate)
|
||||
status: { in: ['downloaded', 'available'] },
|
||||
deletedAt: null,
|
||||
},
|
||||
@@ -181,11 +182,12 @@ export async function POST(request: NextRequest) {
|
||||
logger.debug(`Updated audiobook ${audiobookRecord.id} with year: ${year || 'unchanged'}, series: ${series || 'unchanged'}`);
|
||||
}
|
||||
|
||||
// Check if user already has an active (non-deleted) request for this audiobook
|
||||
// Check if user already has an active (non-deleted) audiobook request for this audiobook
|
||||
const existingRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
userId: req.user.id,
|
||||
audiobookId: audiobookRecord.id,
|
||||
type: 'audiobook', // Only check audiobook requests (ebook requests are separate)
|
||||
deletedAt: null, // Only check active requests
|
||||
},
|
||||
});
|
||||
@@ -263,6 +265,7 @@ export async function POST(request: NextRequest) {
|
||||
userId: req.user.id,
|
||||
audiobookId: audiobookRecord.id,
|
||||
status: 'awaiting_approval',
|
||||
type: 'audiobook', // Explicit type for user-created requests
|
||||
progress: 0,
|
||||
selectedTorrent: torrent as any, // Store the selected torrent for later
|
||||
},
|
||||
@@ -304,6 +307,7 @@ export async function POST(request: NextRequest) {
|
||||
userId: req.user.id,
|
||||
audiobookId: audiobookRecord.id,
|
||||
status: 'downloading',
|
||||
type: 'audiobook', // Explicit type for user-created requests
|
||||
progress: 0,
|
||||
},
|
||||
include: {
|
||||
|
||||
@@ -136,6 +136,8 @@ async function handler(req: AuthenticatedRequest) {
|
||||
where: {
|
||||
userId,
|
||||
audiobookId: audiobook.id,
|
||||
type: 'audiobook', // Only check audiobook requests (ebook requests are separate)
|
||||
deletedAt: null, // Only check active requests
|
||||
},
|
||||
});
|
||||
|
||||
@@ -187,6 +189,7 @@ async function handler(req: AuthenticatedRequest) {
|
||||
userId,
|
||||
audiobookId: audiobook.id,
|
||||
status: initialStatus,
|
||||
type: 'audiobook', // Explicit type for user-created requests
|
||||
priority: 0,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -2,16 +2,13 @@
|
||||
* Component: Fetch E-book API
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*
|
||||
* Triggers e-book download for a completed request
|
||||
* Creates an ebook request for a completed audiobook request
|
||||
*/
|
||||
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
import { requireAuth, requireAdmin, AuthenticatedRequest } from '@/lib/middleware/auth';
|
||||
import { prisma } from '@/lib/db';
|
||||
import { downloadEbook } from '@/lib/services/ebook-scraper';
|
||||
import { buildAudiobookPath } from '@/lib/utils/file-organizer';
|
||||
import fs from 'fs/promises';
|
||||
import path from 'path';
|
||||
import { getJobQueueService } from '@/lib/services/job-queue.service';
|
||||
import { RMABLogger } from '@/lib/utils/logger';
|
||||
|
||||
const logger = RMABLogger.create('API.FetchEbook');
|
||||
@@ -23,7 +20,7 @@ export async function POST(
|
||||
return requireAuth(request, async (req: AuthenticatedRequest) => {
|
||||
return requireAdmin(req, async () => {
|
||||
try {
|
||||
const { id } = await params;
|
||||
const { id: parentRequestId } = await params;
|
||||
|
||||
// Check if e-book sidecar is enabled
|
||||
const ebookEnabledConfig = await prisma.configuration.findUnique({
|
||||
@@ -37,118 +34,108 @@ export async function POST(
|
||||
);
|
||||
}
|
||||
|
||||
// Get the request with audiobook data
|
||||
const requestRecord = await prisma.request.findUnique({
|
||||
where: { id },
|
||||
// Get the parent request with audiobook data
|
||||
const parentRequest = await prisma.request.findUnique({
|
||||
where: { id: parentRequestId },
|
||||
include: {
|
||||
audiobook: true,
|
||||
},
|
||||
});
|
||||
|
||||
if (!requestRecord) {
|
||||
if (!parentRequest) {
|
||||
return NextResponse.json(
|
||||
{ error: 'Request not found' },
|
||||
{ status: 404 }
|
||||
);
|
||||
}
|
||||
|
||||
// Check if request is in completed state
|
||||
if (!['downloaded', 'available'].includes(requestRecord.status)) {
|
||||
// Check if parent request is in completed state
|
||||
if (!['downloaded', 'available'].includes(parentRequest.status)) {
|
||||
return NextResponse.json(
|
||||
{ error: `Cannot fetch e-book for request in ${requestRecord.status} status` },
|
||||
{ error: `Cannot fetch e-book for request in ${parentRequest.status} status` },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
const audiobook = requestRecord.audiobook;
|
||||
|
||||
// Get configuration
|
||||
const [mediaDirConfig, templateConfig, formatConfig, baseUrlConfig, flaresolverrConfig] = await Promise.all([
|
||||
prisma.configuration.findUnique({ where: { key: 'media_dir' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'audiobook_path_template' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_preferred_format' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_base_url' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_flaresolverr_url' } }),
|
||||
]);
|
||||
|
||||
const mediaDir = mediaDirConfig?.value || '/media/audiobooks';
|
||||
const template = templateConfig?.value || '{author}/{title} {asin}';
|
||||
const preferredFormat = formatConfig?.value || 'epub';
|
||||
const baseUrl = baseUrlConfig?.value || 'https://annas-archive.li';
|
||||
const flaresolverrUrl = flaresolverrConfig?.value || undefined;
|
||||
|
||||
// Fetch year from audible cache if ASIN is available
|
||||
let year: number | undefined;
|
||||
if (audiobook.audibleAsin) {
|
||||
const audibleCache = await prisma.audibleCache.findUnique({
|
||||
where: { asin: audiobook.audibleAsin },
|
||||
select: { releaseDate: true },
|
||||
});
|
||||
if (audibleCache?.releaseDate) {
|
||||
year = new Date(audibleCache.releaseDate).getFullYear();
|
||||
}
|
||||
}
|
||||
|
||||
// Build target path using centralized function
|
||||
const targetPath = buildAudiobookPath(
|
||||
mediaDir,
|
||||
template,
|
||||
{
|
||||
author: audiobook.author,
|
||||
title: audiobook.title,
|
||||
narrator: audiobook.narrator || undefined,
|
||||
asin: audiobook.audibleAsin || undefined,
|
||||
year,
|
||||
}
|
||||
);
|
||||
|
||||
logger.debug('Fetch e-book request', {
|
||||
requestId: id,
|
||||
title: audiobook.title,
|
||||
author: audiobook.author,
|
||||
targetPath,
|
||||
format: preferredFormat,
|
||||
baseUrl,
|
||||
flaresolverr: flaresolverrUrl || 'none'
|
||||
// Check if an ebook request already exists for this parent
|
||||
const existingEbookRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
parentRequestId,
|
||||
type: 'ebook',
|
||||
deletedAt: null,
|
||||
},
|
||||
});
|
||||
|
||||
// Check if target directory exists
|
||||
try {
|
||||
await fs.access(targetPath);
|
||||
} catch {
|
||||
logger.debug(`Target directory not found: ${targetPath}`);
|
||||
return NextResponse.json(
|
||||
{ error: 'Audiobook directory not found. Was the audiobook properly organized?' },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
if (existingEbookRequest) {
|
||||
// Check status - if failed/pending, we can retry
|
||||
if (['failed', 'awaiting_search'].includes(existingEbookRequest.status)) {
|
||||
// Reset and retry
|
||||
await prisma.request.update({
|
||||
where: { id: existingEbookRequest.id },
|
||||
data: {
|
||||
status: 'pending',
|
||||
progress: 0,
|
||||
errorMessage: null,
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
// Download e-book
|
||||
const result = await downloadEbook(
|
||||
audiobook.audibleAsin || '',
|
||||
audiobook.title,
|
||||
audiobook.author,
|
||||
targetPath,
|
||||
preferredFormat,
|
||||
baseUrl,
|
||||
undefined, // No logger in API context
|
||||
flaresolverrUrl
|
||||
);
|
||||
// Trigger search job
|
||||
const jobQueue = getJobQueueService();
|
||||
await jobQueue.addSearchEbookJob(existingEbookRequest.id, {
|
||||
id: parentRequest.audiobook.id,
|
||||
title: parentRequest.audiobook.title,
|
||||
author: parentRequest.audiobook.author,
|
||||
asin: parentRequest.audiobook.audibleAsin || undefined,
|
||||
});
|
||||
|
||||
if (result.success) {
|
||||
logger.info(`E-book downloaded: ${result.filePath ? path.basename(result.filePath) : 'unknown'} for "${audiobook.title}"`);
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
message: `E-book downloaded: ${result.filePath ? path.basename(result.filePath) : 'unknown'}`,
|
||||
format: result.format,
|
||||
});
|
||||
} else {
|
||||
logger.warn(`E-book download failed for "${audiobook.title}"`, { error: result.error });
|
||||
logger.info(`Retrying ebook request ${existingEbookRequest.id} for "${parentRequest.audiobook.title}"`);
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
message: 'E-book search retried',
|
||||
requestId: existingEbookRequest.id,
|
||||
});
|
||||
}
|
||||
|
||||
// Already exists and not in a retryable state
|
||||
return NextResponse.json({
|
||||
success: false,
|
||||
message: result.error || 'E-book download failed',
|
||||
message: `E-book request already exists (status: ${existingEbookRequest.status})`,
|
||||
requestId: existingEbookRequest.id,
|
||||
});
|
||||
}
|
||||
|
||||
// Create new ebook request
|
||||
const ebookRequest = await prisma.request.create({
|
||||
data: {
|
||||
userId: parentRequest.userId,
|
||||
audiobookId: parentRequest.audiobookId,
|
||||
type: 'ebook',
|
||||
parentRequestId,
|
||||
status: 'pending',
|
||||
progress: 0,
|
||||
},
|
||||
});
|
||||
|
||||
logger.info(`Created ebook request ${ebookRequest.id} for "${parentRequest.audiobook.title}"`);
|
||||
|
||||
// Trigger ebook search job
|
||||
const jobQueue = getJobQueueService();
|
||||
await jobQueue.addSearchEbookJob(ebookRequest.id, {
|
||||
id: parentRequest.audiobook.id,
|
||||
title: parentRequest.audiobook.title,
|
||||
author: parentRequest.audiobook.author,
|
||||
asin: parentRequest.audiobook.audibleAsin || undefined,
|
||||
});
|
||||
|
||||
logger.info(`Triggered search_ebook job for request ${ebookRequest.id}`);
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
message: 'E-book request created and search started',
|
||||
requestId: ebookRequest.id,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Unexpected error', { error: error instanceof Error ? error.message : String(error) });
|
||||
return NextResponse.json(
|
||||
|
||||
@@ -45,13 +45,14 @@ export async function POST(request: NextRequest) {
|
||||
const body = await req.json();
|
||||
const { audiobook } = CreateRequestSchema.parse(body);
|
||||
|
||||
// First check: Is there an existing request in 'downloaded' or 'available' status?
|
||||
// First check: Is there an existing audiobook request in 'downloaded' or 'available' status?
|
||||
// This catches the gap where files are organized but Plex hasn't scanned yet
|
||||
const existingActiveRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
audiobook: {
|
||||
audibleAsin: audiobook.asin,
|
||||
},
|
||||
type: 'audiobook', // Only check audiobook requests (ebook requests are separate)
|
||||
status: { in: ['downloaded', 'available'] },
|
||||
deletedAt: null,
|
||||
},
|
||||
@@ -165,11 +166,12 @@ export async function POST(request: NextRequest) {
|
||||
logger.debug(`Updated audiobook ${audiobookRecord.id} with year: ${year || 'unchanged'}, series: ${series || 'unchanged'}`);
|
||||
}
|
||||
|
||||
// Check if user already has an active (non-deleted) request for this audiobook
|
||||
// Check if user already has an active (non-deleted) audiobook request for this audiobook
|
||||
const existingRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
userId: req.user.id,
|
||||
audiobookId: audiobookRecord.id,
|
||||
type: 'audiobook', // Only check audiobook requests (ebook requests are separate)
|
||||
deletedAt: null, // Only check active requests
|
||||
},
|
||||
});
|
||||
@@ -257,6 +259,7 @@ export async function POST(request: NextRequest) {
|
||||
userId: req.user.id,
|
||||
audiobookId: audiobookRecord.id,
|
||||
status: initialStatus,
|
||||
type: 'audiobook', // Explicit type for user-created requests
|
||||
progress: 0,
|
||||
},
|
||||
include: {
|
||||
@@ -353,6 +356,7 @@ export async function GET(request: NextRequest) {
|
||||
const status = searchParams.get('status');
|
||||
const limit = parseInt(searchParams.get('limit') || '50', 10);
|
||||
const myOnly = searchParams.get('myOnly') === 'true';
|
||||
const type = searchParams.get('type'); // 'audiobook', 'ebook', or null for all
|
||||
const isAdmin = req.user.role === 'admin';
|
||||
|
||||
// Build query
|
||||
@@ -362,6 +366,10 @@ export async function GET(request: NextRequest) {
|
||||
if (status) {
|
||||
where.status = status;
|
||||
}
|
||||
// Filter by type if specified (otherwise returns all types)
|
||||
if (type && ['audiobook', 'ebook'].includes(type)) {
|
||||
where.type = type;
|
||||
}
|
||||
// Only show active (non-deleted) requests
|
||||
where.deletedAt = null;
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ import { InteractiveTorrentSearchModal } from './InteractiveTorrentSearchModal';
|
||||
interface RequestCardProps {
|
||||
request: {
|
||||
id: string;
|
||||
type?: 'audiobook' | 'ebook';
|
||||
status: string;
|
||||
progress: number;
|
||||
errorMessage?: string;
|
||||
@@ -38,10 +39,14 @@ export function RequestCard({ request, showActions = true }: RequestCardProps) {
|
||||
const [showError, setShowError] = React.useState(false);
|
||||
const [showInteractiveSearch, setShowInteractiveSearch] = React.useState(false);
|
||||
|
||||
const requestType = request.type || 'audiobook';
|
||||
const isEbook = requestType === 'ebook';
|
||||
|
||||
const canCancel = ['pending', 'searching', 'downloading'].includes(request.status);
|
||||
const isActive = ['searching', 'downloading', 'processing'].includes(request.status);
|
||||
const isFailed = request.status === 'failed';
|
||||
const canSearch = ['pending', 'failed', 'awaiting_search'].includes(request.status);
|
||||
// Ebook requests don't support interactive search (Anna's Archive only)
|
||||
const canSearch = !isEbook && ['pending', 'failed', 'awaiting_search'].includes(request.status);
|
||||
|
||||
const handleCancel = async () => {
|
||||
if (window.confirm('Are you sure you want to cancel this request?')) {
|
||||
@@ -100,19 +105,30 @@ export function RequestCard({ request, showActions = true }: RequestCardProps) {
|
||||
/>
|
||||
) : (
|
||||
<div className="w-full h-full flex items-center justify-center">
|
||||
<svg
|
||||
className="w-12 h-12 text-gray-400"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
viewBox="0 0 24 24"
|
||||
>
|
||||
<path
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
strokeWidth={2}
|
||||
d="M9 19V6l12-3v13M9 19c0 1.105-1.343 2-3 2s-3-.895-3-2 1.343-2 3-2 3 .895 3 2zm12-3c0 1.105-1.343 2-3 2s-3-.895-3-2 1.343-2 3-2 3 .895 3 2zM9 10l12-3"
|
||||
/>
|
||||
</svg>
|
||||
{isEbook ? (
|
||||
<svg
|
||||
className="w-12 h-12"
|
||||
style={{ color: '#f16f19' }}
|
||||
fill="currentColor"
|
||||
viewBox="0 0 24 24"
|
||||
>
|
||||
<path d="M6 2c-1.1 0-1.99.9-1.99 2L4 20c0 1.1.89 2 1.99 2H18c1.1 0 2-.9 2-2V8l-6-6H6zm7 7V3.5L18.5 9H13z" />
|
||||
</svg>
|
||||
) : (
|
||||
<svg
|
||||
className="w-12 h-12 text-gray-400"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
viewBox="0 0 24 24"
|
||||
>
|
||||
<path
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
strokeWidth={2}
|
||||
d="M9 19V6l12-3v13M9 19c0 1.105-1.343 2-3 2s-3-.895-3-2 1.343-2 3-2 3 .895 3 2zm12-3c0 1.105-1.343 2-3 2s-3-.895-3-2 1.343-2 3-2 3 .895 3 2zM9 10l12-3"
|
||||
/>
|
||||
</svg>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
@@ -130,9 +146,20 @@ export function RequestCard({ request, showActions = true }: RequestCardProps) {
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Status Badge */}
|
||||
<div className="flex items-center gap-2">
|
||||
{/* Status Badge and Type Badge */}
|
||||
<div className="flex items-center gap-2 flex-wrap">
|
||||
<StatusBadge status={request.status} progress={request.progress} />
|
||||
{isEbook && (
|
||||
<span
|
||||
className="inline-flex items-center gap-1 px-2 py-0.5 text-xs font-medium rounded-full"
|
||||
style={{ backgroundColor: '#f16f1920', color: '#f16f19' }}
|
||||
>
|
||||
<svg className="w-3 h-3" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path d="M9 4.804A7.968 7.968 0 005.5 4c-1.255 0-2.443.29-3.5.804v10A7.969 7.969 0 015.5 14c1.669 0 3.218.51 4.5 1.385A7.962 7.962 0 0114.5 14c1.255 0 2.443.29 3.5.804v-10A7.968 7.968 0 0014.5 4c-1.255 0-2.443.29-3.5.804V12a1 1 0 11-2 0V4.804z" />
|
||||
</svg>
|
||||
Ebook
|
||||
</span>
|
||||
)}
|
||||
{isActive && request.progress > 0 && (
|
||||
<div className="flex items-center gap-1 text-xs text-gray-500 dark:text-gray-400">
|
||||
<div className="animate-pulse w-2 h-2 bg-blue-500 rounded-full"></div>
|
||||
|
||||
@@ -927,7 +927,7 @@ export async function isInLibrary(
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if book has already been requested
|
||||
* Check if book has already been requested (audiobook request)
|
||||
* @param userId - User ID
|
||||
* @param asin - Audible ASIN
|
||||
* @returns true if book is already requested
|
||||
@@ -939,6 +939,8 @@ export async function isAlreadyRequested(
|
||||
const request = await prisma.request.findFirst({
|
||||
where: {
|
||||
userId,
|
||||
type: 'audiobook', // Only check audiobook requests (ebook requests are separate)
|
||||
deletedAt: null, // Only check active requests
|
||||
audiobook: {
|
||||
audibleAsin: asin,
|
||||
},
|
||||
|
||||
@@ -44,12 +44,14 @@ export async function processCleanupSeededTorrents(payload: CleanupSeededTorrent
|
||||
|
||||
logger.info(`Loaded configuration for ${indexerConfigMap.size} indexers`);
|
||||
|
||||
// Find all completed requests + soft-deleted requests (orphaned downloads)
|
||||
// Find all completed audiobook requests + soft-deleted audiobook requests (orphaned downloads)
|
||||
// IMPORTANT: Only cleanup requests that are truly complete and not being actively processed
|
||||
// NOTE: Multiple requests can share the same torrent hash (e.g., re-requesting same audiobook)
|
||||
// Before deleting torrent, we check if other active requests are using it
|
||||
// NOTE: Ebook requests use direct HTTP downloads (no torrent seeding), so they're excluded
|
||||
const completedRequests = await prisma.request.findMany({
|
||||
where: {
|
||||
type: 'audiobook', // Only audiobook requests (ebooks don't have torrents to seed)
|
||||
OR: [
|
||||
// Active requests that are fully available (scanned by Plex/ABS)
|
||||
{
|
||||
@@ -148,11 +150,12 @@ export async function processCleanupSeededTorrents(payload: CleanupSeededTorrent
|
||||
|
||||
logger.info(`Torrent ${torrent.name} (${indexerName}) has met seeding requirement (${Math.floor(actualSeedingTime / 60)}/${seedingConfig.seedingTimeMinutes} minutes)`);
|
||||
|
||||
// CRITICAL: Check if any other active (non-deleted) request is using this same torrent hash
|
||||
// CRITICAL: Check if any other active (non-deleted) audiobook request is using this same torrent hash
|
||||
// This prevents deleting shared torrents when user re-requests the same audiobook
|
||||
const otherActiveRequests = await prisma.request.findMany({
|
||||
where: {
|
||||
id: { not: request.id }, // Exclude current request
|
||||
type: 'audiobook', // Only check audiobook requests
|
||||
deletedAt: null, // Only check active requests
|
||||
downloadHistory: {
|
||||
some: {
|
||||
|
||||
@@ -0,0 +1,504 @@
|
||||
/**
|
||||
* Component: Direct Download Job Processors
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*
|
||||
* Handles direct HTTP downloads for ebooks from Anna's Archive.
|
||||
* Reports progress similar to qBittorrent/SABnzbd for unified UI.
|
||||
*/
|
||||
|
||||
import { StartDirectDownloadPayload, MonitorDirectDownloadPayload, getJobQueueService } from '../services/job-queue.service';
|
||||
import { prisma } from '../db';
|
||||
import { getConfigService } from '../services/config.service';
|
||||
import { RMABLogger } from '../utils/logger';
|
||||
import { extractDownloadUrl, ExtractedDownload } from '../services/ebook-scraper';
|
||||
import axios from 'axios';
|
||||
import fs from 'fs/promises';
|
||||
import { createWriteStream } from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
const DOWNLOAD_TIMEOUT_MS = 120000; // 2 minutes per download attempt
|
||||
const MAX_DOWNLOAD_ATTEMPTS = 5;
|
||||
const PROGRESS_UPDATE_INTERVAL_MS = 2000; // Update progress every 2 seconds
|
||||
|
||||
// In-memory tracking for active downloads
|
||||
interface ActiveDownload {
|
||||
id: string;
|
||||
requestId: string;
|
||||
downloadHistoryId: string;
|
||||
targetPath: string;
|
||||
bytesDownloaded: number;
|
||||
bytesTotal: number;
|
||||
startTime: number;
|
||||
lastUpdateTime: number;
|
||||
completed: boolean;
|
||||
failed: boolean;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
const activeDownloads = new Map<string, ActiveDownload>();
|
||||
|
||||
/**
|
||||
* Generate unique download ID
|
||||
*/
|
||||
function generateDownloadId(): string {
|
||||
return `dl_${Date.now()}_${Math.random().toString(36).substring(2, 9)}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Process start direct download job
|
||||
* Initiates the HTTP download and schedules monitoring
|
||||
*/
|
||||
export async function processStartDirectDownload(payload: StartDirectDownloadPayload): Promise<any> {
|
||||
const { requestId, downloadHistoryId, downloadUrl, targetFilename, expectedSize, jobId } = payload;
|
||||
|
||||
const logger = RMABLogger.forJob(jobId, 'DirectDownload');
|
||||
|
||||
logger.info(`Starting direct download for request ${requestId}`);
|
||||
|
||||
try {
|
||||
// Update request status to downloading
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'downloading',
|
||||
progress: 0,
|
||||
downloadAttempts: { increment: 1 },
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
// Update download history
|
||||
await prisma.downloadHistory.update({
|
||||
where: { id: downloadHistoryId },
|
||||
data: {
|
||||
downloadStatus: 'downloading',
|
||||
startedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
// Get download configuration
|
||||
const configService = getConfigService();
|
||||
const downloadsDir = await configService.get('downloads_dir') || '/downloads';
|
||||
const baseUrl = await configService.get('ebook_sidecar_base_url') || 'https://annas-archive.li';
|
||||
const preferredFormat = await configService.get('ebook_sidecar_preferred_format') || 'epub';
|
||||
const flaresolverrUrl = await configService.get('ebook_sidecar_flaresolverr_url') || undefined;
|
||||
|
||||
// Get all download URLs from download history (stored as JSON in torrentUrl)
|
||||
const downloadHistory = await prisma.downloadHistory.findUnique({
|
||||
where: { id: downloadHistoryId },
|
||||
});
|
||||
|
||||
let downloadUrls: string[] = [];
|
||||
try {
|
||||
downloadUrls = downloadHistory?.torrentUrl ? JSON.parse(downloadHistory.torrentUrl) : [downloadUrl];
|
||||
} catch {
|
||||
downloadUrls = [downloadUrl];
|
||||
}
|
||||
|
||||
logger.info(`Have ${downloadUrls.length} download URL(s) to try`);
|
||||
|
||||
// Try each slow download URL until one succeeds
|
||||
let downloadResult: { success: boolean; filePath?: string; format?: string; error?: string } = {
|
||||
success: false,
|
||||
error: 'No download URLs available',
|
||||
};
|
||||
|
||||
const attemptsLimit = Math.min(downloadUrls.length, MAX_DOWNLOAD_ATTEMPTS);
|
||||
|
||||
for (let i = 0; i < attemptsLimit; i++) {
|
||||
const slowLink = downloadUrls[i];
|
||||
logger.info(`Attempting download link ${i + 1}/${attemptsLimit}...`);
|
||||
|
||||
try {
|
||||
// Extract actual download URL from slow download page
|
||||
const extracted = await extractDownloadUrl(
|
||||
slowLink,
|
||||
baseUrl,
|
||||
preferredFormat,
|
||||
logger,
|
||||
flaresolverrUrl
|
||||
);
|
||||
|
||||
if (!extracted) {
|
||||
logger.warn(`No download URL found on page ${i + 1}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
logger.info(`Downloading from: ${new URL(extracted.url).host} (format: ${extracted.format})`);
|
||||
|
||||
// Build target path with actual format
|
||||
const sanitizedFilename = sanitizeFilename(`${targetFilename.replace(/\.[^.]+$/, '')}.${extracted.format}`);
|
||||
const targetPath = path.join(downloadsDir, sanitizedFilename);
|
||||
|
||||
// Create download tracking entry
|
||||
const downloadId = generateDownloadId();
|
||||
const downloadEntry: ActiveDownload = {
|
||||
id: downloadId,
|
||||
requestId,
|
||||
downloadHistoryId,
|
||||
targetPath,
|
||||
bytesDownloaded: 0,
|
||||
bytesTotal: expectedSize || 0,
|
||||
startTime: Date.now(),
|
||||
lastUpdateTime: Date.now(),
|
||||
completed: false,
|
||||
failed: false,
|
||||
};
|
||||
activeDownloads.set(downloadId, downloadEntry);
|
||||
|
||||
// Start download with progress tracking
|
||||
const success = await downloadFileWithProgress(
|
||||
extracted.url,
|
||||
targetPath,
|
||||
downloadEntry,
|
||||
logger
|
||||
);
|
||||
|
||||
if (success) {
|
||||
downloadResult = {
|
||||
success: true,
|
||||
filePath: targetPath,
|
||||
format: extracted.format,
|
||||
};
|
||||
|
||||
// Get final file size
|
||||
try {
|
||||
const stats = await fs.stat(targetPath);
|
||||
downloadEntry.bytesTotal = stats.size;
|
||||
downloadEntry.bytesDownloaded = stats.size;
|
||||
} catch {
|
||||
// Ignore stat errors
|
||||
}
|
||||
|
||||
logger.info(`Download completed: ${sanitizedFilename}`);
|
||||
break;
|
||||
}
|
||||
|
||||
logger.warn(`Download attempt ${i + 1} failed`);
|
||||
activeDownloads.delete(downloadId);
|
||||
} catch (error) {
|
||||
logger.warn(`Download link ${i + 1} error: ${error instanceof Error ? error.message : 'Unknown'}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (!downloadResult.success) {
|
||||
// All attempts failed
|
||||
logger.error(`All ${attemptsLimit} download attempts failed`);
|
||||
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'failed',
|
||||
errorMessage: downloadResult.error || 'All download attempts failed',
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.downloadHistory.update({
|
||||
where: { id: downloadHistoryId },
|
||||
data: {
|
||||
downloadStatus: 'failed',
|
||||
downloadError: downloadResult.error || 'All download attempts failed',
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
success: false,
|
||||
message: 'Download failed',
|
||||
requestId,
|
||||
error: downloadResult.error,
|
||||
};
|
||||
}
|
||||
|
||||
// Download succeeded - update records and trigger organize
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'processing',
|
||||
progress: 100,
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.downloadHistory.update({
|
||||
where: { id: downloadHistoryId },
|
||||
data: {
|
||||
downloadStatus: 'completed',
|
||||
completedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
// Get audiobook ID for organize job
|
||||
const request = await prisma.request.findUnique({
|
||||
where: { id: requestId },
|
||||
include: { audiobook: true },
|
||||
});
|
||||
|
||||
if (!request) {
|
||||
throw new Error('Request not found after download');
|
||||
}
|
||||
|
||||
// Trigger organize files job
|
||||
const jobQueue = getJobQueueService();
|
||||
await jobQueue.addOrganizeJob(
|
||||
requestId,
|
||||
request.audiobookId,
|
||||
downloadResult.filePath!
|
||||
);
|
||||
|
||||
logger.info(`Download complete, triggered organize job for ${downloadResult.filePath}`);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
message: 'Download completed, organizing files',
|
||||
requestId,
|
||||
filePath: downloadResult.filePath,
|
||||
format: downloadResult.format,
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error(`Error: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'failed',
|
||||
errorMessage: error instanceof Error ? error.message : 'Unknown error during download',
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.downloadHistory.update({
|
||||
where: { id: downloadHistoryId },
|
||||
data: {
|
||||
downloadStatus: 'failed',
|
||||
downloadError: error instanceof Error ? error.message : 'Unknown error',
|
||||
},
|
||||
});
|
||||
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Download file with progress tracking
|
||||
*/
|
||||
async function downloadFileWithProgress(
|
||||
url: string,
|
||||
targetPath: string,
|
||||
tracking: ActiveDownload,
|
||||
logger: RMABLogger
|
||||
): Promise<boolean> {
|
||||
try {
|
||||
// Ensure target directory exists
|
||||
await fs.mkdir(path.dirname(targetPath), { recursive: true });
|
||||
|
||||
// Start download with axios streaming
|
||||
const response = await axios({
|
||||
method: 'GET',
|
||||
url,
|
||||
responseType: 'stream',
|
||||
timeout: DOWNLOAD_TIMEOUT_MS,
|
||||
headers: {
|
||||
'User-Agent': 'ReadMeABook/1.0 (Audiobook Automation)',
|
||||
},
|
||||
});
|
||||
|
||||
// Get content length if available
|
||||
const contentLength = parseInt(response.headers['content-length'] || '0', 10);
|
||||
if (contentLength > 0) {
|
||||
tracking.bytesTotal = contentLength;
|
||||
}
|
||||
|
||||
// Create write stream
|
||||
const writer = createWriteStream(targetPath);
|
||||
|
||||
// Track progress
|
||||
let bytesDownloaded = 0;
|
||||
let lastLogTime = Date.now();
|
||||
let lastDbUpdateTime = Date.now();
|
||||
|
||||
response.data.on('data', (chunk: Buffer) => {
|
||||
bytesDownloaded += chunk.length;
|
||||
tracking.bytesDownloaded = bytesDownloaded;
|
||||
tracking.lastUpdateTime = Date.now();
|
||||
|
||||
// Log and update database every 2 seconds
|
||||
const now = Date.now();
|
||||
if (now - lastLogTime >= 2000) {
|
||||
const percent = tracking.bytesTotal > 0
|
||||
? Math.round((bytesDownloaded / tracking.bytesTotal) * 100)
|
||||
: 0;
|
||||
const speedMBps = bytesDownloaded / ((now - tracking.startTime) / 1000) / (1024 * 1024);
|
||||
logger.info(`Download progress: ${percent}% (${(bytesDownloaded / (1024 * 1024)).toFixed(1)} MB, ${speedMBps.toFixed(2)} MB/s)`);
|
||||
lastLogTime = now;
|
||||
|
||||
// Update database with progress (non-blocking)
|
||||
if (now - lastDbUpdateTime >= PROGRESS_UPDATE_INTERVAL_MS) {
|
||||
lastDbUpdateTime = now;
|
||||
|
||||
// Non-blocking update - fire and forget
|
||||
prisma.request.update({
|
||||
where: { id: tracking.requestId },
|
||||
data: {
|
||||
progress: Math.min(percent, 99), // Cap at 99% until fully complete
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
}).catch(() => {}); // Ignore errors during progress update
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Pipe to file
|
||||
response.data.pipe(writer);
|
||||
|
||||
// Wait for completion
|
||||
return new Promise((resolve, reject) => {
|
||||
writer.on('finish', () => {
|
||||
tracking.completed = true;
|
||||
resolve(true);
|
||||
});
|
||||
|
||||
writer.on('error', (error) => {
|
||||
tracking.failed = true;
|
||||
tracking.error = error.message;
|
||||
reject(error);
|
||||
});
|
||||
|
||||
response.data.on('error', (error: Error) => {
|
||||
tracking.failed = true;
|
||||
tracking.error = error.message;
|
||||
writer.close();
|
||||
// Clean up partial file
|
||||
fs.unlink(targetPath).catch(() => {});
|
||||
reject(error);
|
||||
});
|
||||
});
|
||||
} catch (error) {
|
||||
tracking.failed = true;
|
||||
tracking.error = error instanceof Error ? error.message : 'Unknown error';
|
||||
|
||||
// Clean up partial file
|
||||
try {
|
||||
await fs.unlink(targetPath);
|
||||
} catch {
|
||||
// Ignore cleanup errors
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Process monitor direct download job
|
||||
* Checks download progress and updates database
|
||||
* Note: For direct downloads, most tracking happens in processStartDirectDownload
|
||||
* This is kept for potential future use with async downloads
|
||||
*/
|
||||
export async function processMonitorDirectDownload(payload: MonitorDirectDownloadPayload): Promise<any> {
|
||||
const { requestId, downloadHistoryId, downloadId, targetPath, expectedSize, jobId } = payload;
|
||||
|
||||
const logger = RMABLogger.forJob(jobId, 'MonitorDirectDownload');
|
||||
|
||||
// Check if download is tracked
|
||||
const download = activeDownloads.get(downloadId);
|
||||
|
||||
if (!download) {
|
||||
// Download not in memory - check file existence
|
||||
try {
|
||||
const stats = await fs.stat(targetPath);
|
||||
logger.info(`Download file exists: ${targetPath} (${stats.size} bytes)`);
|
||||
|
||||
// If file exists and is complete, assume success
|
||||
if (expectedSize && stats.size >= expectedSize) {
|
||||
return {
|
||||
success: true,
|
||||
completed: true,
|
||||
message: 'Download already completed',
|
||||
requestId,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
// File doesn't exist
|
||||
}
|
||||
|
||||
logger.warn(`Download ${downloadId} not found in tracking`);
|
||||
return {
|
||||
success: false,
|
||||
message: 'Download not found',
|
||||
requestId,
|
||||
};
|
||||
}
|
||||
|
||||
// Update database with progress
|
||||
const progress = download.bytesTotal > 0
|
||||
? Math.min(99, Math.round((download.bytesDownloaded / download.bytesTotal) * 100))
|
||||
: 0;
|
||||
|
||||
const elapsed = Date.now() - download.startTime;
|
||||
const speed = elapsed > 0 ? download.bytesDownloaded / (elapsed / 1000) : 0;
|
||||
const eta = speed > 0 && download.bytesTotal > download.bytesDownloaded
|
||||
? Math.round((download.bytesTotal - download.bytesDownloaded) / speed)
|
||||
: 0;
|
||||
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
progress,
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
if (download.completed) {
|
||||
logger.info(`Download ${downloadId} completed`);
|
||||
return {
|
||||
success: true,
|
||||
completed: true,
|
||||
requestId,
|
||||
bytesDownloaded: download.bytesDownloaded,
|
||||
bytesTotal: download.bytesTotal,
|
||||
};
|
||||
}
|
||||
|
||||
if (download.failed) {
|
||||
logger.error(`Download ${downloadId} failed: ${download.error}`);
|
||||
return {
|
||||
success: false,
|
||||
completed: false,
|
||||
requestId,
|
||||
error: download.error,
|
||||
};
|
||||
}
|
||||
|
||||
// Still in progress - schedule another monitor
|
||||
const jobQueue = getJobQueueService();
|
||||
await jobQueue.addMonitorDirectDownloadJob(
|
||||
requestId,
|
||||
downloadHistoryId,
|
||||
downloadId,
|
||||
targetPath,
|
||||
expectedSize,
|
||||
PROGRESS_UPDATE_INTERVAL_MS / 1000
|
||||
);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
completed: false,
|
||||
requestId,
|
||||
progress,
|
||||
speed,
|
||||
eta,
|
||||
bytesDownloaded: download.bytesDownloaded,
|
||||
bytesTotal: download.bytesTotal,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Sanitize filename for filesystem
|
||||
*/
|
||||
function sanitizeFilename(filename: string): string {
|
||||
return filename
|
||||
.replace(/[<>:"/\\|?*]/g, '') // Remove invalid chars
|
||||
.replace(/\s+/g, ' ') // Collapse spaces
|
||||
.trim()
|
||||
.substring(0, 200); // Limit length
|
||||
}
|
||||
@@ -57,9 +57,11 @@ export async function processMonitorRssFeeds(payload: MonitorRssFeedsPayload): P
|
||||
return { success: true, message: 'No RSS results', matched: 0 };
|
||||
}
|
||||
|
||||
// Get all active requests awaiting search (missing audiobooks)
|
||||
// Get all active audiobook requests awaiting search (missing audiobooks)
|
||||
// Note: RSS feeds are for torrents, so only audiobook requests are matched
|
||||
const missingRequests = await prisma.request.findMany({
|
||||
where: {
|
||||
type: 'audiobook', // Only audiobook requests (RSS feeds are for torrents)
|
||||
status: 'awaiting_search',
|
||||
deletedAt: null,
|
||||
},
|
||||
|
||||
@@ -14,6 +14,7 @@ import { generateFilesHash } from '../utils/files-hash';
|
||||
/**
|
||||
* Process organize files job
|
||||
* Moves completed downloads to media library in proper directory structure
|
||||
* Handles both audiobook and ebook request types with appropriate branching
|
||||
*/
|
||||
export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promise<any> {
|
||||
const { requestId, audiobookId, downloadPath, jobId } = payload;
|
||||
@@ -24,6 +25,27 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi
|
||||
logger.info(`Download path: ${downloadPath}`);
|
||||
|
||||
try {
|
||||
// Fetch request to determine type
|
||||
const request = await prisma.request.findUnique({
|
||||
where: { id: requestId },
|
||||
include: {
|
||||
user: { select: { plexUsername: true } },
|
||||
},
|
||||
});
|
||||
|
||||
if (!request) {
|
||||
throw new Error(`Request ${requestId} not found`);
|
||||
}
|
||||
|
||||
const requestType = request.type || 'audiobook'; // Default to audiobook for backward compatibility
|
||||
logger.info(`Request type: ${requestType}`);
|
||||
|
||||
// Branch based on request type
|
||||
if (requestType === 'ebook') {
|
||||
return await processEbookOrganization(payload, request, logger);
|
||||
}
|
||||
|
||||
// Continue with audiobook organization flow
|
||||
// Update request status to processing
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
@@ -149,6 +171,10 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi
|
||||
errors: result.errors,
|
||||
});
|
||||
|
||||
// Create ebook request if ebook downloads enabled (for audiobook requests only)
|
||||
// This replaces the old inline ebook sidecar download
|
||||
await createEbookRequestIfEnabled(requestId, audiobook, request.userId, result.targetPath, logger);
|
||||
|
||||
// Trigger filesystem scan if enabled (Plex or Audiobookshelf)
|
||||
const configService = getConfigService();
|
||||
const backendMode = await configService.getBackendMode();
|
||||
@@ -433,3 +459,215 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// =========================================================================
|
||||
// EBOOK-SPECIFIC ORGANIZATION
|
||||
// =========================================================================
|
||||
|
||||
/**
|
||||
* Process ebook organization (simplified flow compared to audiobooks)
|
||||
* - No metadata tagging
|
||||
* - No cover art download
|
||||
* - No files hash generation
|
||||
* - Sends "available" notification at downloaded state (terminal for ebooks)
|
||||
*/
|
||||
async function processEbookOrganization(
|
||||
payload: OrganizeFilesPayload,
|
||||
request: { id: string; userId: string; type: string; user: { plexUsername: string | null } },
|
||||
logger: RMABLogger
|
||||
): Promise<any> {
|
||||
const { requestId, audiobookId, downloadPath, jobId } = payload;
|
||||
|
||||
logger.info(`Processing ebook organization for request ${requestId}`);
|
||||
|
||||
// Update request status to processing
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'processing',
|
||||
progress: 100,
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
// Get book details (works for both audiobooks and ebooks)
|
||||
const book = await prisma.audiobook.findUnique({
|
||||
where: { id: audiobookId },
|
||||
});
|
||||
|
||||
if (!book) {
|
||||
throw new Error(`Book ${audiobookId} not found`);
|
||||
}
|
||||
|
||||
logger.info(`Organizing ebook: ${book.title} by ${book.author}`);
|
||||
|
||||
// Get file organizer and template
|
||||
const organizer = await getFileOrganizer();
|
||||
const templateConfig = await prisma.configuration.findUnique({
|
||||
where: { key: 'audiobook_path_template' },
|
||||
});
|
||||
const template = templateConfig?.value || '{author}/{title} {asin}';
|
||||
|
||||
// Organize ebook files (organizer will detect ebook type and skip audio-specific processing)
|
||||
const result = await organizer.organizeEbook(
|
||||
downloadPath,
|
||||
{
|
||||
title: book.title,
|
||||
author: book.author,
|
||||
asin: book.audibleAsin || undefined,
|
||||
year: book.year || undefined,
|
||||
},
|
||||
template,
|
||||
jobId ? { jobId, context: 'FileOrganizer.Ebook' } : undefined
|
||||
);
|
||||
|
||||
if (!result.success) {
|
||||
throw new Error(`Ebook organization failed: ${result.errors.join(', ')}`);
|
||||
}
|
||||
|
||||
logger.info(`Successfully moved ebook to ${result.targetPath}`);
|
||||
|
||||
// Update book record with file path
|
||||
await prisma.audiobook.update({
|
||||
where: { id: audiobookId },
|
||||
data: {
|
||||
filePath: result.targetPath,
|
||||
fileFormat: result.format || 'epub',
|
||||
status: 'completed',
|
||||
completedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
// Update request to downloaded (terminal state for ebooks)
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'downloaded',
|
||||
progress: 100,
|
||||
completedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
logger.info(`Ebook request ${requestId} completed - status: downloaded (terminal)`);
|
||||
|
||||
// Send "available" notification for ebooks at downloaded state
|
||||
// (since ebooks don't transition to 'available' via Plex matching)
|
||||
const jobQueue = getJobQueueService();
|
||||
await jobQueue.addNotificationJob(
|
||||
'request_available',
|
||||
requestId,
|
||||
book.title,
|
||||
book.author,
|
||||
request.user.plexUsername || 'Unknown User'
|
||||
).catch((error) => {
|
||||
logger.error('Failed to queue notification', { error: error instanceof Error ? error.message : String(error) });
|
||||
});
|
||||
|
||||
// Trigger filesystem scan if enabled (same as audiobooks)
|
||||
const configService = getConfigService();
|
||||
const backendMode = await configService.getBackendMode();
|
||||
const configKey = backendMode === 'audiobookshelf'
|
||||
? 'audiobookshelf.trigger_scan_after_import'
|
||||
: 'plex.trigger_scan_after_import';
|
||||
const scanEnabled = await configService.get(configKey);
|
||||
|
||||
logger.debug(`Ebook library scan check: backendMode=${backendMode}, configKey=${configKey}, scanEnabled=${scanEnabled}`);
|
||||
|
||||
if (scanEnabled === 'true') {
|
||||
try {
|
||||
const libraryService = await getLibraryService();
|
||||
const libraryId = backendMode === 'audiobookshelf'
|
||||
? await configService.get('audiobookshelf.library_id')
|
||||
: await configService.get('plex_audiobook_library_id');
|
||||
|
||||
if (libraryId) {
|
||||
await libraryService.triggerLibraryScan(libraryId);
|
||||
logger.info(`Triggered ${backendMode} filesystem scan for library ${libraryId}`);
|
||||
} else {
|
||||
logger.warn(`Library ID not configured for ${backendMode}, skipping scan`);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(`Failed to trigger filesystem scan: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
}
|
||||
} else {
|
||||
logger.debug(`Ebook library scan disabled (scanEnabled=${scanEnabled})`);
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
message: 'Ebook organized successfully',
|
||||
requestId,
|
||||
audiobookId,
|
||||
targetPath: result.targetPath,
|
||||
format: result.format,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Create ebook request if ebook downloads are enabled
|
||||
* Called after audiobook organization completes
|
||||
*/
|
||||
async function createEbookRequestIfEnabled(
|
||||
parentRequestId: string,
|
||||
audiobook: { id: string; title: string; author: string; audibleAsin: string | null },
|
||||
userId: string,
|
||||
targetPath: string,
|
||||
logger: RMABLogger
|
||||
): Promise<void> {
|
||||
try {
|
||||
// Check if ebook downloads are enabled
|
||||
const configService = getConfigService();
|
||||
const ebookEnabled = await configService.get('ebook_sidecar_enabled');
|
||||
|
||||
if (ebookEnabled !== 'true') {
|
||||
logger.info('Ebook downloads disabled, skipping ebook request creation');
|
||||
return;
|
||||
}
|
||||
|
||||
// Check if an ebook request already exists for this parent
|
||||
const existingEbookRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
parentRequestId,
|
||||
type: 'ebook',
|
||||
deletedAt: null,
|
||||
},
|
||||
});
|
||||
|
||||
if (existingEbookRequest) {
|
||||
logger.info(`Ebook request already exists for parent ${parentRequestId}: ${existingEbookRequest.id}`);
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info(`Creating ebook request for "${audiobook.title}" (parent: ${parentRequestId})`);
|
||||
|
||||
// Create new ebook request (auto-approved since parent was approved)
|
||||
const ebookRequest = await prisma.request.create({
|
||||
data: {
|
||||
userId,
|
||||
audiobookId: audiobook.id,
|
||||
type: 'ebook',
|
||||
parentRequestId,
|
||||
status: 'pending', // Will trigger search_ebook job
|
||||
progress: 0,
|
||||
},
|
||||
});
|
||||
|
||||
logger.info(`Created ebook request ${ebookRequest.id}`);
|
||||
|
||||
// Trigger ebook search job
|
||||
const jobQueue = getJobQueueService();
|
||||
await jobQueue.addSearchEbookJob(ebookRequest.id, {
|
||||
id: audiobook.id,
|
||||
title: audiobook.title,
|
||||
author: audiobook.author,
|
||||
asin: audiobook.audibleAsin || undefined,
|
||||
});
|
||||
|
||||
logger.info(`Triggered search_ebook job for request ${ebookRequest.id}`);
|
||||
} catch (error) {
|
||||
// Don't fail the main audiobook organization if ebook request creation fails
|
||||
logger.error(`Failed to create ebook request: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -249,9 +249,11 @@ export async function processPlexRecentlyAddedCheck(payload: PlexRecentlyAddedPa
|
||||
}
|
||||
}
|
||||
|
||||
// Check for all non-terminal requests to match
|
||||
// Check for all non-terminal audiobook requests to match
|
||||
// Note: Ebook requests don't match to Plex/ABS library - they stop at 'downloaded' status
|
||||
const matchableRequests = await prisma.request.findMany({
|
||||
where: {
|
||||
type: 'audiobook', // Only match audiobook requests (ebooks don't go to 'available')
|
||||
status: { notIn: ['available', 'cancelled'] },
|
||||
deletedAt: null,
|
||||
},
|
||||
|
||||
@@ -37,9 +37,11 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
|
||||
localPath: pathMappingConfig.download_client_local_path || '',
|
||||
};
|
||||
|
||||
// Find all active requests in awaiting_import status
|
||||
// Find all active audiobook requests in awaiting_import status
|
||||
// Note: Ebook requests use the same organize_files processor but with type branching
|
||||
const requests = await prisma.request.findMany({
|
||||
where: {
|
||||
type: 'audiobook', // Only audiobook requests (ebooks handled by same processor but different flow)
|
||||
status: 'awaiting_import',
|
||||
deletedAt: null,
|
||||
},
|
||||
|
||||
@@ -21,9 +21,11 @@ export async function processRetryMissingTorrents(payload: RetryMissingTorrentsP
|
||||
logger.info('Starting retry job for requests awaiting search...');
|
||||
|
||||
try {
|
||||
// Find all active requests in awaiting_search status
|
||||
// Find all active audiobook requests in awaiting_search status
|
||||
// Note: Ebook requests have separate search mechanism (search_ebook job)
|
||||
const requests = await prisma.request.findMany({
|
||||
where: {
|
||||
type: 'audiobook', // Only audiobook requests (ebooks use different search)
|
||||
status: 'awaiting_search',
|
||||
deletedAt: null,
|
||||
},
|
||||
|
||||
@@ -433,10 +433,12 @@ export async function processScanPlex(payload: ScanPlexPayload): Promise<any> {
|
||||
logger.info(`No orphaned audiobooks found`);
|
||||
}
|
||||
|
||||
// 6. Match all non-terminal requests against library
|
||||
// 6. Match all non-terminal audiobook requests against library
|
||||
// Note: Ebook requests don't match to Plex/ABS library - they stop at 'downloaded' status
|
||||
logger.info(`Checking for matchable requests...`);
|
||||
const matchableRequests = await prisma.request.findMany({
|
||||
where: {
|
||||
type: 'audiobook', // Only match audiobook requests (ebooks don't go to 'available')
|
||||
status: { notIn: ['available', 'cancelled'] },
|
||||
deletedAt: null,
|
||||
},
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
/**
|
||||
* Component: Search Ebook Job Processor
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*
|
||||
* Searches Anna's Archive for ebook downloads.
|
||||
* Part of the first-class ebook request flow.
|
||||
*/
|
||||
|
||||
import { SearchEbookPayload, EbookSearchResult, getJobQueueService } from '../services/job-queue.service';
|
||||
import { prisma } from '../db';
|
||||
import { getConfigService } from '../services/config.service';
|
||||
import { RMABLogger } from '../utils/logger';
|
||||
|
||||
// Import ebook scraper functions (we'll refactor these to be reusable)
|
||||
import {
|
||||
searchByAsin,
|
||||
searchByTitle,
|
||||
getSlowDownloadLinks,
|
||||
} from '../services/ebook-scraper';
|
||||
|
||||
/**
|
||||
* Process search ebook job
|
||||
* Searches Anna's Archive for ebook matching the audiobook
|
||||
*/
|
||||
export async function processSearchEbook(payload: SearchEbookPayload): Promise<any> {
|
||||
const { requestId, audiobook, preferredFormat: payloadFormat, jobId } = payload;
|
||||
|
||||
const logger = RMABLogger.forJob(jobId, 'SearchEbook');
|
||||
|
||||
logger.info(`Processing ebook request ${requestId} for "${audiobook.title}"`);
|
||||
|
||||
try {
|
||||
// Update request status to searching
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'searching',
|
||||
searchAttempts: { increment: 1 },
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
// Get ebook configuration
|
||||
const configService = getConfigService();
|
||||
const preferredFormat = payloadFormat || await configService.get('ebook_sidecar_preferred_format') || 'epub';
|
||||
const baseUrl = await configService.get('ebook_sidecar_base_url') || 'https://annas-archive.li';
|
||||
const flaresolverrUrl = await configService.get('ebook_sidecar_flaresolverr_url') || undefined;
|
||||
|
||||
if (flaresolverrUrl) {
|
||||
logger.info(`Using FlareSolverr at ${flaresolverrUrl}`);
|
||||
}
|
||||
|
||||
let md5: string | null = null;
|
||||
let searchMethod: 'asin' | 'title' = 'title';
|
||||
|
||||
// Step 1: Try ASIN search (exact match - best)
|
||||
if (audiobook.asin) {
|
||||
logger.info(`Searching by ASIN: ${audiobook.asin} (format: ${preferredFormat})...`);
|
||||
md5 = await searchByAsin(audiobook.asin, preferredFormat, baseUrl, logger, flaresolverrUrl);
|
||||
|
||||
if (md5) {
|
||||
logger.info(`Found via ASIN: ${md5}`);
|
||||
searchMethod = 'asin';
|
||||
} else {
|
||||
logger.info(`No results for ASIN, falling back to title + author search...`);
|
||||
}
|
||||
}
|
||||
|
||||
// Step 2: Fallback to title + author search
|
||||
if (!md5) {
|
||||
logger.info(`Searching by title + author: "${audiobook.title}" by ${audiobook.author}...`);
|
||||
md5 = await searchByTitle(audiobook.title, audiobook.author, preferredFormat, baseUrl, logger, flaresolverrUrl);
|
||||
|
||||
if (md5) {
|
||||
logger.info(`Found via title search: ${md5}`);
|
||||
searchMethod = 'title';
|
||||
}
|
||||
}
|
||||
|
||||
if (!md5) {
|
||||
// No results found - queue for re-search instead of failing
|
||||
logger.warn(`No ebook found for request ${requestId}, marking as awaiting_search`);
|
||||
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'awaiting_search',
|
||||
errorMessage: 'No ebook found on Anna\'s Archive. Will retry automatically.',
|
||||
lastSearchAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
success: false,
|
||||
message: 'No ebook found, queued for re-search',
|
||||
requestId,
|
||||
};
|
||||
}
|
||||
|
||||
logger.info(`Found MD5: ${md5}`);
|
||||
|
||||
// Step 3: Get slow download links
|
||||
const slowLinks = await getSlowDownloadLinks(md5, baseUrl, logger, flaresolverrUrl);
|
||||
|
||||
if (slowLinks.length === 0) {
|
||||
logger.warn(`No download links available for MD5: ${md5}`);
|
||||
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'awaiting_search',
|
||||
errorMessage: 'Found ebook but no download links available. Will retry automatically.',
|
||||
lastSearchAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
success: false,
|
||||
message: 'No download links available, queued for re-search',
|
||||
requestId,
|
||||
};
|
||||
}
|
||||
|
||||
logger.info(`Found ${slowLinks.length} download link(s)`);
|
||||
|
||||
// Create ebook search result
|
||||
// Note: For future multi-source ranking, this would be one of many results
|
||||
const searchResult: EbookSearchResult = {
|
||||
md5,
|
||||
title: audiobook.title,
|
||||
author: audiobook.author,
|
||||
format: preferredFormat,
|
||||
downloadUrls: slowLinks,
|
||||
source: 'annas_archive',
|
||||
score: searchMethod === 'asin' ? 100 : 80, // ASIN match = higher confidence
|
||||
};
|
||||
|
||||
// TODO: Future enhancement - when indexer support is added for ebooks:
|
||||
// 1. Search Prowlarr for ebook results (filtered to ebook categories)
|
||||
// 2. Rank results using rankEbookResults() with inverted size scoring
|
||||
// 3. Anna's Archive results should get priority bonus to come out on top
|
||||
// For now, Anna's Archive is the only source and always wins.
|
||||
|
||||
logger.info(`==================== EBOOK SEARCH RESULT ====================`);
|
||||
logger.info(`Title: "${audiobook.title}"`);
|
||||
logger.info(`Author: "${audiobook.author}"`);
|
||||
logger.info(`Match Method: ${searchMethod === 'asin' ? 'ASIN (exact)' : 'Title + Author (fuzzy)'}`);
|
||||
logger.info(`Format: ${preferredFormat}`);
|
||||
logger.info(`MD5: ${md5}`);
|
||||
logger.info(`Download Links: ${slowLinks.length}`);
|
||||
logger.info(`Score: ${searchResult.score}/100`);
|
||||
logger.info(`==============================================================`);
|
||||
|
||||
// Create download history record
|
||||
const downloadHistory = await prisma.downloadHistory.create({
|
||||
data: {
|
||||
requestId,
|
||||
indexerName: 'Anna\'s Archive',
|
||||
torrentName: `${audiobook.title} - ${audiobook.author}.${preferredFormat}`,
|
||||
torrentSizeBytes: null, // Unknown until download starts
|
||||
qualityScore: searchResult.score,
|
||||
selected: true,
|
||||
downloadClient: 'direct', // Direct HTTP download
|
||||
downloadStatus: 'queued',
|
||||
},
|
||||
});
|
||||
|
||||
// Trigger direct download job with the best (only) result
|
||||
const jobQueue = getJobQueueService();
|
||||
|
||||
// The first slow link will be tried; if it fails, the processor will try others
|
||||
await jobQueue.addStartDirectDownloadJob(
|
||||
requestId,
|
||||
downloadHistory.id,
|
||||
slowLinks[0], // Start with first link
|
||||
`${audiobook.title} - ${audiobook.author}.${preferredFormat}`,
|
||||
undefined // Size unknown
|
||||
);
|
||||
|
||||
// Store all download URLs in download history for retry purposes
|
||||
await prisma.downloadHistory.update({
|
||||
where: { id: downloadHistory.id },
|
||||
data: {
|
||||
// Store additional URLs in torrentUrl field (JSON array)
|
||||
torrentUrl: JSON.stringify(slowLinks),
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
message: `Found ebook via ${searchMethod === 'asin' ? 'ASIN' : 'title search'}, starting download`,
|
||||
requestId,
|
||||
searchResult: {
|
||||
md5: searchResult.md5,
|
||||
format: searchResult.format,
|
||||
score: searchResult.score,
|
||||
downloadLinksCount: slowLinks.length,
|
||||
},
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error(`Error: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
status: 'failed',
|
||||
errorMessage: error instanceof Error ? error.message : 'Unknown error during ebook search',
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -304,8 +304,9 @@ export async function downloadEbook(
|
||||
|
||||
/**
|
||||
* Step 1: Search Anna's Archive by ASIN and extract MD5 hash
|
||||
* Exported for use by search-ebook processor
|
||||
*/
|
||||
async function searchByAsin(
|
||||
export async function searchByAsin(
|
||||
asin: string,
|
||||
format: string,
|
||||
baseUrl: string,
|
||||
@@ -394,8 +395,9 @@ async function searchByAsin(
|
||||
|
||||
/**
|
||||
* Search Anna's Archive by title and author (fallback method)
|
||||
* Exported for use by search-ebook processor
|
||||
*/
|
||||
async function searchByTitle(
|
||||
export async function searchByTitle(
|
||||
title: string,
|
||||
author: string,
|
||||
format: string,
|
||||
@@ -486,8 +488,9 @@ async function searchByTitle(
|
||||
|
||||
/**
|
||||
* Step 3: Get slow download links from MD5 page (no waitlist only)
|
||||
* Exported for use by search-ebook processor
|
||||
*/
|
||||
async function getSlowDownloadLinks(
|
||||
export async function getSlowDownloadLinks(
|
||||
md5: string,
|
||||
baseUrl: string,
|
||||
logger?: RMABLogger,
|
||||
@@ -561,7 +564,7 @@ async function getSlowDownloadLinks(
|
||||
}
|
||||
}
|
||||
|
||||
interface ExtractedDownload {
|
||||
export interface ExtractedDownload {
|
||||
url: string;
|
||||
format: string;
|
||||
}
|
||||
@@ -570,8 +573,9 @@ interface ExtractedDownload {
|
||||
* Step 4: Extract actual download URL from slow download page
|
||||
* IMPORTANT: Supports dynamic file formats (not hardcoded to .epub)
|
||||
* Returns both URL and detected format
|
||||
* Exported for use by direct-download processor
|
||||
*/
|
||||
async function extractDownloadUrl(
|
||||
export async function extractDownloadUrl(
|
||||
slowDownloadUrl: string,
|
||||
baseUrl: string,
|
||||
format: string,
|
||||
|
||||
@@ -24,7 +24,11 @@ export type JobType =
|
||||
| 'retry_failed_imports'
|
||||
| 'cleanup_seeded_torrents'
|
||||
| 'monitor_rss_feeds'
|
||||
| 'send_notification';
|
||||
| 'send_notification'
|
||||
// Ebook-specific job types
|
||||
| 'search_ebook'
|
||||
| 'start_direct_download'
|
||||
| 'monitor_direct_download';
|
||||
|
||||
export interface JobPayload {
|
||||
jobId?: string; // Database job ID (added automatically by addJob)
|
||||
@@ -95,6 +99,45 @@ export interface CleanupSeededTorrentsPayload extends JobPayload {
|
||||
scheduledJobId?: string;
|
||||
}
|
||||
|
||||
// Ebook-specific payload interfaces
|
||||
export interface SearchEbookPayload extends JobPayload {
|
||||
requestId: string;
|
||||
audiobook: {
|
||||
id: string;
|
||||
title: string;
|
||||
author: string;
|
||||
asin?: string; // ASIN for Anna's Archive search (best match)
|
||||
};
|
||||
preferredFormat?: string; // epub, pdf, mobi, azw3 (default: from config)
|
||||
}
|
||||
|
||||
export interface EbookSearchResult {
|
||||
md5: string;
|
||||
title: string;
|
||||
author: string;
|
||||
format: string;
|
||||
fileSize?: number;
|
||||
downloadUrls: string[]; // Slow download URLs from Anna's Archive
|
||||
source: 'annas_archive'; // For future indexer support
|
||||
score: number; // Ranking score (for future multi-source ranking)
|
||||
}
|
||||
|
||||
export interface StartDirectDownloadPayload extends JobPayload {
|
||||
requestId: string;
|
||||
downloadHistoryId: string;
|
||||
downloadUrl: string;
|
||||
targetFilename: string;
|
||||
expectedSize?: number;
|
||||
}
|
||||
|
||||
export interface MonitorDirectDownloadPayload extends JobPayload {
|
||||
requestId: string;
|
||||
downloadHistoryId: string;
|
||||
downloadId: string; // Internal tracking ID
|
||||
targetPath: string; // Full path to the downloading file
|
||||
expectedSize?: number;
|
||||
}
|
||||
|
||||
export interface SendNotificationPayload extends JobPayload {
|
||||
event: 'request_pending_approval' | 'request_approved' | 'request_available' | 'request_error';
|
||||
requestId: string;
|
||||
@@ -301,6 +344,22 @@ export class JobQueueService {
|
||||
const { processSendNotification } = await import('../processors/send-notification.processor');
|
||||
return await processSendNotification(job.data);
|
||||
});
|
||||
|
||||
// Ebook-specific processors
|
||||
this.queue.process('search_ebook', 3, async (job: BullJob<SearchEbookPayload>) => {
|
||||
const { processSearchEbook } = await import('../processors/search-ebook.processor');
|
||||
return await processSearchEbook(job.data);
|
||||
});
|
||||
|
||||
this.queue.process('start_direct_download', 3, async (job: BullJob<StartDirectDownloadPayload>) => {
|
||||
const { processStartDirectDownload } = await import('../processors/direct-download.processor');
|
||||
return await processStartDirectDownload(job.data);
|
||||
});
|
||||
|
||||
this.queue.process('monitor_direct_download', 5, async (job: BullJob<MonitorDirectDownloadPayload>) => {
|
||||
const { processMonitorDirectDownload } = await import('../processors/direct-download.processor');
|
||||
return await processMonitorDirectDownload(job.data);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -635,6 +694,83 @@ export class JobQueueService {
|
||||
);
|
||||
}
|
||||
|
||||
// =========================================================================
|
||||
// EBOOK-SPECIFIC JOB METHODS
|
||||
// =========================================================================
|
||||
|
||||
/**
|
||||
* Add search ebook job (Anna's Archive search)
|
||||
*/
|
||||
async addSearchEbookJob(
|
||||
requestId: string,
|
||||
audiobook: { id: string; title: string; author: string; asin?: string },
|
||||
preferredFormat?: string
|
||||
): Promise<string> {
|
||||
return await this.addJob(
|
||||
'search_ebook',
|
||||
{
|
||||
requestId,
|
||||
audiobook,
|
||||
preferredFormat,
|
||||
} as SearchEbookPayload,
|
||||
{
|
||||
priority: 10, // High priority for user-initiated requests
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Add start direct download job (HTTP download for ebooks)
|
||||
*/
|
||||
async addStartDirectDownloadJob(
|
||||
requestId: string,
|
||||
downloadHistoryId: string,
|
||||
downloadUrl: string,
|
||||
targetFilename: string,
|
||||
expectedSize?: number
|
||||
): Promise<string> {
|
||||
return await this.addJob(
|
||||
'start_direct_download',
|
||||
{
|
||||
requestId,
|
||||
downloadHistoryId,
|
||||
downloadUrl,
|
||||
targetFilename,
|
||||
expectedSize,
|
||||
} as StartDirectDownloadPayload,
|
||||
{
|
||||
priority: 9, // High priority - download selected ebook
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Add monitor direct download job (tracks HTTP download progress)
|
||||
*/
|
||||
async addMonitorDirectDownloadJob(
|
||||
requestId: string,
|
||||
downloadHistoryId: string,
|
||||
downloadId: string,
|
||||
targetPath: string,
|
||||
expectedSize?: number,
|
||||
delaySeconds: number = 0
|
||||
): Promise<string> {
|
||||
return await this.addJob(
|
||||
'monitor_direct_download',
|
||||
{
|
||||
requestId,
|
||||
downloadHistoryId,
|
||||
downloadId,
|
||||
targetPath,
|
||||
expectedSize,
|
||||
} as MonitorDirectDownloadPayload,
|
||||
{
|
||||
priority: 5, // Medium priority
|
||||
delay: delaySeconds * 1000,
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get job by ID
|
||||
*/
|
||||
|
||||
@@ -26,7 +26,7 @@ export interface DeleteRequestResult {
|
||||
/**
|
||||
* Soft delete a request with intelligent cleanup of media files and torrents
|
||||
*
|
||||
* Logic:
|
||||
* Logic (audiobook requests):
|
||||
* 1. Check if request exists and is not already deleted
|
||||
* 2. For each download:
|
||||
* - If unlimited seeding (0): Log and keep seeding, no monitoring
|
||||
@@ -34,7 +34,15 @@ export interface DeleteRequestResult {
|
||||
* - If seeding requirement met: Delete torrent + files
|
||||
* - If still seeding: Keep in qBittorrent for cleanup job
|
||||
* 3. Delete media files (title folder only)
|
||||
* 4. Soft delete request (set deletedAt, deletedBy)
|
||||
* 4. Delete from backend library (Plex/ABS)
|
||||
* 5. Clear audiobook availability linkage
|
||||
* 6. Soft delete request (set deletedAt, deletedBy)
|
||||
*
|
||||
* Logic (ebook requests):
|
||||
* 1. Check if request exists and is not already deleted
|
||||
* 2. Delete ebook files only (leave audiobook files intact)
|
||||
* 3. Soft delete request (set deletedAt, deletedBy)
|
||||
* Note: No backend library deletion or audiobook linkage clearing for ebooks
|
||||
*/
|
||||
export async function deleteRequest(
|
||||
requestId: string,
|
||||
@@ -57,6 +65,7 @@ export async function deleteRequest(
|
||||
audibleAsin: true,
|
||||
plexGuid: true,
|
||||
absItemId: true,
|
||||
fileFormat: true,
|
||||
},
|
||||
},
|
||||
downloadHistory: {
|
||||
@@ -71,6 +80,10 @@ export async function deleteRequest(
|
||||
},
|
||||
});
|
||||
|
||||
// Determine request type (default to audiobook for backward compatibility)
|
||||
const requestType = (request as any)?.type || 'audiobook';
|
||||
const isEbook = requestType === 'ebook';
|
||||
|
||||
if (!request) {
|
||||
return {
|
||||
success: false,
|
||||
@@ -87,10 +100,11 @@ export async function deleteRequest(
|
||||
let torrentsKeptSeeding = 0;
|
||||
let torrentsKeptUnlimited = 0;
|
||||
|
||||
// 2. Handle downloads & seeding
|
||||
// 2. Handle downloads & seeding (skip for ebooks - they use direct HTTP downloads)
|
||||
const downloadHistory = request.downloadHistory[0];
|
||||
const skipTorrentHandling = isEbook; // Ebooks use direct downloads, not torrents/NZBs
|
||||
|
||||
if (downloadHistory && downloadHistory.indexerName) {
|
||||
if (!skipTorrentHandling && downloadHistory && downloadHistory.indexerName) {
|
||||
try {
|
||||
// Get indexer seeding configuration
|
||||
const { getConfigService } = await import('./config.service');
|
||||
@@ -186,7 +200,9 @@ export async function deleteRequest(
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Delete media files (title folder only)
|
||||
// 3. Delete media files
|
||||
// For audiobooks: delete entire title folder
|
||||
// For ebooks: delete only ebook files (leave audiobook files intact)
|
||||
let filesDeleted = false;
|
||||
try {
|
||||
const { getConfigService } = await import('./config.service');
|
||||
@@ -219,15 +235,34 @@ export async function deleteRequest(
|
||||
}
|
||||
);
|
||||
|
||||
// Check if folder exists and delete it
|
||||
// Check if folder exists
|
||||
try {
|
||||
await fs.access(titleFolderPath);
|
||||
|
||||
// Delete the title folder (not the author folder)
|
||||
await fs.rm(titleFolderPath, { recursive: true, force: true });
|
||||
if (isEbook) {
|
||||
// For ebooks: only delete ebook files, leave audiobook files intact
|
||||
const ebookExtensions = ['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.fb2', '.cbz', '.cbr'];
|
||||
const files = await fs.readdir(titleFolderPath);
|
||||
|
||||
logger.info(`Deleted media directory: ${titleFolderPath}`);
|
||||
filesDeleted = true;
|
||||
let deletedCount = 0;
|
||||
for (const file of files) {
|
||||
const ext = path.extname(file).toLowerCase();
|
||||
if (ebookExtensions.includes(ext)) {
|
||||
const filePath = path.join(titleFolderPath, file);
|
||||
await fs.unlink(filePath);
|
||||
logger.info(`Deleted ebook file: ${file}`);
|
||||
deletedCount++;
|
||||
}
|
||||
}
|
||||
|
||||
filesDeleted = deletedCount > 0;
|
||||
logger.info(`Deleted ${deletedCount} ebook file(s) from: ${titleFolderPath}`);
|
||||
} else {
|
||||
// For audiobooks: delete the entire title folder
|
||||
await fs.rm(titleFolderPath, { recursive: true, force: true });
|
||||
logger.info(`Deleted media directory: ${titleFolderPath}`);
|
||||
filesDeleted = true;
|
||||
}
|
||||
} catch (accessError) {
|
||||
// Folder doesn't exist - that's okay
|
||||
logger.info(`Media directory not found: ${titleFolderPath}`);
|
||||
@@ -242,143 +277,188 @@ export async function deleteRequest(
|
||||
}
|
||||
|
||||
// 4. Delete from plex_library table and clear audiobook availability
|
||||
// Skip for ebooks - audiobook files and library entry should remain intact
|
||||
// This ensures the book immediately shows as NOT available when searching
|
||||
try {
|
||||
const { getConfigService } = await import('./config.service');
|
||||
const configService = getConfigService();
|
||||
const backendMode = await configService.getBackendMode();
|
||||
if (!isEbook) {
|
||||
try {
|
||||
const { getConfigService } = await import('./config.service');
|
||||
const configService = getConfigService();
|
||||
const backendMode = await configService.getBackendMode();
|
||||
|
||||
// Delete from library backend (ABS or Plex)
|
||||
if (backendMode === 'audiobookshelf' && request.audiobook.absItemId) {
|
||||
// Audiobookshelf: delete the library item from ABS
|
||||
try {
|
||||
const { deleteABSItem } = await import('../services/audiobookshelf/api');
|
||||
await deleteABSItem(request.audiobook.absItemId);
|
||||
logger.info(
|
||||
`Deleted Audiobookshelf library item ${request.audiobook.absItemId} for "${request.audiobook.title}"`
|
||||
);
|
||||
} catch (absError) {
|
||||
logger.error(
|
||||
`Error deleting Audiobookshelf library item ${request.audiobook.absItemId}`,
|
||||
{ error: absError instanceof Error ? absError.message : String(absError) }
|
||||
);
|
||||
// Continue with deletion even if ABS deletion fails
|
||||
// Delete from library backend (ABS or Plex)
|
||||
if (backendMode === 'audiobookshelf' && request.audiobook.absItemId) {
|
||||
// Audiobookshelf: delete the library item from ABS
|
||||
try {
|
||||
const { deleteABSItem } = await import('../services/audiobookshelf/api');
|
||||
await deleteABSItem(request.audiobook.absItemId);
|
||||
logger.info(
|
||||
`Deleted Audiobookshelf library item ${request.audiobook.absItemId} for "${request.audiobook.title}"`
|
||||
);
|
||||
} catch (absError) {
|
||||
logger.error(
|
||||
`Error deleting Audiobookshelf library item ${request.audiobook.absItemId}`,
|
||||
{ error: absError instanceof Error ? absError.message : String(absError) }
|
||||
);
|
||||
// Continue with deletion even if ABS deletion fails
|
||||
}
|
||||
} else if (backendMode === 'plex' && request.audiobook.plexGuid) {
|
||||
// Plex: delete the library item from Plex by ratingKey
|
||||
try {
|
||||
// Query plex_library table to get the ratingKey
|
||||
const plexLibraryRecord = await prisma.plexLibrary.findUnique({
|
||||
where: { plexGuid: request.audiobook.plexGuid },
|
||||
select: { plexRatingKey: true },
|
||||
});
|
||||
|
||||
if (plexLibraryRecord && plexLibraryRecord.plexRatingKey) {
|
||||
const ratingKey = plexLibraryRecord.plexRatingKey;
|
||||
|
||||
// Get Plex config
|
||||
const plexServerUrl = (await configService.get('plex_url')) || '';
|
||||
const plexToken = (await configService.get('plex_token')) || '';
|
||||
|
||||
if (plexServerUrl && plexToken) {
|
||||
const { getPlexService } = await import('../integrations/plex.service');
|
||||
const plexService = getPlexService();
|
||||
await plexService.deleteItem(plexServerUrl, plexToken, ratingKey);
|
||||
logger.info(
|
||||
`Deleted Plex library item ${ratingKey} (plexGuid: ${request.audiobook.plexGuid}) for "${request.audiobook.title}"`
|
||||
);
|
||||
} else {
|
||||
logger.warn('Plex server URL or token not configured, skipping Plex library deletion');
|
||||
}
|
||||
} else {
|
||||
logger.warn(
|
||||
`No plexRatingKey found in plex_library for plexGuid: ${request.audiobook.plexGuid}`
|
||||
);
|
||||
}
|
||||
} catch (plexError) {
|
||||
logger.error(
|
||||
`Error deleting Plex library item (plexGuid: ${request.audiobook.plexGuid})`,
|
||||
{ error: plexError instanceof Error ? plexError.message : String(plexError) }
|
||||
);
|
||||
// Continue with deletion even if Plex deletion fails
|
||||
}
|
||||
}
|
||||
} else if (backendMode === 'plex' && request.audiobook.plexGuid) {
|
||||
// Plex: delete the library item from Plex by ratingKey
|
||||
|
||||
// Delete ALL plex_library records matching this audiobook's title and author
|
||||
// This handles cases where there might be duplicate library records
|
||||
// and ensures the book doesn't show as "In Your Library" during searches
|
||||
try {
|
||||
// Query plex_library table to get the ratingKey
|
||||
const plexLibraryRecord = await prisma.plexLibrary.findUnique({
|
||||
where: { plexGuid: request.audiobook.plexGuid },
|
||||
select: { plexRatingKey: true },
|
||||
// Find all matching library records (by title/author fuzzy match)
|
||||
const matchingLibraryRecords = await prisma.plexLibrary.findMany({
|
||||
where: {
|
||||
title: {
|
||||
contains: request.audiobook.title.substring(0, 20),
|
||||
mode: 'insensitive',
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
if (plexLibraryRecord && plexLibraryRecord.plexRatingKey) {
|
||||
const ratingKey = plexLibraryRecord.plexRatingKey;
|
||||
// Filter to exact matches (case-insensitive title and author)
|
||||
const exactMatches = matchingLibraryRecords.filter((record) => {
|
||||
const titleMatch = record.title.toLowerCase() === request.audiobook.title.toLowerCase();
|
||||
const authorMatch = record.author.toLowerCase() === request.audiobook.author.toLowerCase();
|
||||
return titleMatch && authorMatch;
|
||||
});
|
||||
|
||||
// Get Plex config
|
||||
const plexServerUrl = (await configService.get('plex_url')) || '';
|
||||
const plexToken = (await configService.get('plex_token')) || '';
|
||||
if (exactMatches.length > 0) {
|
||||
// Delete all exact matches
|
||||
const deletePromises = exactMatches.map((record) =>
|
||||
prisma.plexLibrary.delete({ where: { id: record.id } })
|
||||
);
|
||||
|
||||
if (plexServerUrl && plexToken) {
|
||||
const { getPlexService } = await import('../integrations/plex.service');
|
||||
const plexService = getPlexService();
|
||||
await plexService.deleteItem(plexServerUrl, plexToken, ratingKey);
|
||||
logger.info(
|
||||
`Deleted Plex library item ${ratingKey} (plexGuid: ${request.audiobook.plexGuid}) for "${request.audiobook.title}"`
|
||||
);
|
||||
} else {
|
||||
logger.warn('Plex server URL or token not configured, skipping Plex library deletion');
|
||||
}
|
||||
await Promise.all(deletePromises);
|
||||
|
||||
logger.info(
|
||||
`Deleted ${exactMatches.length} plex_library record(s) for "${request.audiobook.title}"`
|
||||
);
|
||||
} else {
|
||||
logger.warn(
|
||||
`No plexRatingKey found in plex_library for plexGuid: ${request.audiobook.plexGuid}`
|
||||
logger.info(
|
||||
`No plex_library records found for "${request.audiobook.title}"`
|
||||
);
|
||||
}
|
||||
} catch (plexError) {
|
||||
} catch (libError) {
|
||||
logger.error(
|
||||
`Error deleting Plex library item (plexGuid: ${request.audiobook.plexGuid})`,
|
||||
{ error: plexError instanceof Error ? plexError.message : String(plexError) }
|
||||
`Error deleting plex_library records`,
|
||||
{ error: libError instanceof Error ? libError.message : String(libError) }
|
||||
);
|
||||
// Continue with deletion even if Plex deletion fails
|
||||
// Continue with deletion even if library cleanup fails
|
||||
}
|
||||
}
|
||||
|
||||
// Delete ALL plex_library records matching this audiobook's title and author
|
||||
// This handles cases where there might be duplicate library records
|
||||
// and ensures the book doesn't show as "In Your Library" during searches
|
||||
// Clear audiobook record linkage
|
||||
const updateData: any = {
|
||||
status: 'requested', // Reset to requested state
|
||||
updatedAt: new Date(),
|
||||
};
|
||||
|
||||
// Clear library linkage based on backend mode
|
||||
if (backendMode === 'audiobookshelf') {
|
||||
updateData.absItemId = null;
|
||||
} else {
|
||||
updateData.plexGuid = null;
|
||||
}
|
||||
|
||||
await prisma.audiobook.update({
|
||||
where: { id: request.audiobook.id },
|
||||
data: updateData,
|
||||
});
|
||||
|
||||
logger.info(
|
||||
`Cleared availability status for audiobook ${request.audiobook.id}`
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
`Error clearing audiobook status`,
|
||||
{ error: error instanceof Error ? error.message : String(error) }
|
||||
);
|
||||
// Continue with deletion even if this fails
|
||||
}
|
||||
} else {
|
||||
logger.info(`Skipping backend library deletion for ebook request ${requestId}`);
|
||||
}
|
||||
|
||||
// 5. Delete child requests (ebook requests linked to this audiobook request)
|
||||
if (!isEbook) {
|
||||
try {
|
||||
// Find all matching library records (by title/author fuzzy match)
|
||||
const matchingLibraryRecords = await prisma.plexLibrary.findMany({
|
||||
const childRequests = await prisma.request.findMany({
|
||||
where: {
|
||||
title: {
|
||||
contains: request.audiobook.title.substring(0, 20),
|
||||
mode: 'insensitive',
|
||||
},
|
||||
parentRequestId: requestId,
|
||||
deletedAt: null,
|
||||
},
|
||||
select: {
|
||||
id: true,
|
||||
type: true,
|
||||
},
|
||||
});
|
||||
|
||||
// Filter to exact matches (case-insensitive title and author)
|
||||
const exactMatches = matchingLibraryRecords.filter((record) => {
|
||||
const titleMatch = record.title.toLowerCase() === request.audiobook.title.toLowerCase();
|
||||
const authorMatch = record.author.toLowerCase() === request.audiobook.author.toLowerCase();
|
||||
return titleMatch && authorMatch;
|
||||
});
|
||||
if (childRequests.length > 0) {
|
||||
logger.info(`Found ${childRequests.length} child request(s) to delete`);
|
||||
|
||||
if (exactMatches.length > 0) {
|
||||
// Delete all exact matches
|
||||
const deletePromises = exactMatches.map((record) =>
|
||||
prisma.plexLibrary.delete({ where: { id: record.id } })
|
||||
);
|
||||
// Soft delete all child requests
|
||||
await prisma.request.updateMany({
|
||||
where: {
|
||||
parentRequestId: requestId,
|
||||
deletedAt: null,
|
||||
},
|
||||
data: {
|
||||
deletedAt: new Date(),
|
||||
deletedBy: adminUserId,
|
||||
},
|
||||
});
|
||||
|
||||
await Promise.all(deletePromises);
|
||||
|
||||
logger.info(
|
||||
`Deleted ${exactMatches.length} plex_library record(s) for "${request.audiobook.title}"`
|
||||
);
|
||||
} else {
|
||||
logger.info(
|
||||
`No plex_library records found for "${request.audiobook.title}"`
|
||||
);
|
||||
logger.info(`Soft-deleted ${childRequests.length} child request(s)`);
|
||||
}
|
||||
} catch (libError) {
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
`Error deleting plex_library records`,
|
||||
{ error: libError instanceof Error ? libError.message : String(libError) }
|
||||
`Error deleting child requests for ${requestId}`,
|
||||
{ error: error instanceof Error ? error.message : String(error) }
|
||||
);
|
||||
// Continue with deletion even if library cleanup fails
|
||||
// Continue with parent deletion even if child deletion fails
|
||||
}
|
||||
|
||||
// Clear audiobook record linkage
|
||||
const updateData: any = {
|
||||
status: 'requested', // Reset to requested state
|
||||
updatedAt: new Date(),
|
||||
};
|
||||
|
||||
// Clear library linkage based on backend mode
|
||||
if (backendMode === 'audiobookshelf') {
|
||||
updateData.absItemId = null;
|
||||
} else {
|
||||
updateData.plexGuid = null;
|
||||
}
|
||||
|
||||
await prisma.audiobook.update({
|
||||
where: { id: request.audiobook.id },
|
||||
data: updateData,
|
||||
});
|
||||
|
||||
logger.info(
|
||||
`Cleared availability status for audiobook ${request.audiobook.id}`
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
`Error clearing audiobook status`,
|
||||
{ error: error instanceof Error ? error.message : String(error) }
|
||||
);
|
||||
// Continue with deletion even if this fails
|
||||
}
|
||||
|
||||
// 5. Soft delete request
|
||||
// 6. Soft delete request
|
||||
await prisma.request.update({
|
||||
where: { id: requestId },
|
||||
data: {
|
||||
|
||||
+107
-50
@@ -19,7 +19,6 @@ import {
|
||||
checkDiskSpace,
|
||||
} from './chapter-merger';
|
||||
import { prisma } from '../db';
|
||||
import { downloadEbook } from '../services/ebook-scraper';
|
||||
import { substituteTemplate, type TemplateVariables } from './path-template.util';
|
||||
|
||||
export interface AudiobookMetadata {
|
||||
@@ -42,6 +41,13 @@ export interface OrganizationResult {
|
||||
coverArtFile?: string;
|
||||
}
|
||||
|
||||
export interface EbookOrganizationResult {
|
||||
success: boolean;
|
||||
targetPath: string;
|
||||
errors: string[];
|
||||
format?: string;
|
||||
}
|
||||
|
||||
export interface ValidationResult {
|
||||
isValid: boolean;
|
||||
issues: string[];
|
||||
@@ -399,55 +405,10 @@ export class FileOrganizer {
|
||||
}
|
||||
}
|
||||
|
||||
// E-book sidecar: Download accompanying e-book if enabled
|
||||
try {
|
||||
const ebookConfig = await prisma.configuration.findUnique({
|
||||
where: { key: 'ebook_sidecar_enabled' },
|
||||
});
|
||||
|
||||
const ebookEnabled = ebookConfig?.value === 'true';
|
||||
|
||||
if (ebookEnabled) {
|
||||
await logger?.info(`E-book sidecar enabled, searching for e-book...`);
|
||||
|
||||
// Get configuration
|
||||
const [formatConfig, baseUrlConfig, flaresolverrConfig] = await Promise.all([
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_preferred_format' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_base_url' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_flaresolverr_url' } }),
|
||||
]);
|
||||
|
||||
const preferredFormat = formatConfig?.value || 'epub';
|
||||
const baseUrl = baseUrlConfig?.value || 'https://annas-archive.li';
|
||||
const flaresolverrUrl = flaresolverrConfig?.value || undefined;
|
||||
|
||||
// Download e-book (will try ASIN first, then fall back to title+author)
|
||||
const ebookResult = await downloadEbook(
|
||||
audiobook.asin || '', // ASIN (optional - will fallback to title+author if empty)
|
||||
audiobook.title,
|
||||
audiobook.author,
|
||||
targetPath, // Same directory as audiobook
|
||||
preferredFormat,
|
||||
baseUrl,
|
||||
logger ?? undefined,
|
||||
flaresolverrUrl
|
||||
);
|
||||
|
||||
if (ebookResult.success && ebookResult.filePath) {
|
||||
await logger?.info(`E-book downloaded: ${path.basename(ebookResult.filePath)}`);
|
||||
result.filesMovedCount++;
|
||||
} else {
|
||||
await logger?.warn(`E-book download failed: ${ebookResult.error}`);
|
||||
result.errors.push(`E-book sidecar: ${ebookResult.error}`);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
await logger?.warn(
|
||||
`E-book sidecar error: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
);
|
||||
result.errors.push('E-book sidecar failed');
|
||||
// Don't throw - audiobook organization continues
|
||||
}
|
||||
// NOTE: E-book downloads are now handled via first-class ebook requests
|
||||
// The createEbookRequestIfEnabled() function in organize-files.processor.ts
|
||||
// creates a separate ebook request that goes through the full job queue flow.
|
||||
// This replaces the old inline ebook sidecar download that happened here.
|
||||
|
||||
result.targetPath = targetPath;
|
||||
result.success = true;
|
||||
@@ -680,6 +641,102 @@ export class FileOrganizer {
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Organize ebook file into proper directory structure
|
||||
* Simplified compared to audiobooks - no metadata tagging, cover art, or chapter merging
|
||||
*/
|
||||
async organizeEbook(
|
||||
downloadPath: string,
|
||||
metadata: { title: string; author: string; asin?: string; year?: number },
|
||||
template: string,
|
||||
loggerConfig?: LoggerConfig
|
||||
): Promise<EbookOrganizationResult> {
|
||||
const logger = loggerConfig ? RMABLogger.forJob(loggerConfig.jobId, loggerConfig.context) : null;
|
||||
|
||||
const result: EbookOrganizationResult = {
|
||||
success: false,
|
||||
targetPath: '',
|
||||
errors: [],
|
||||
};
|
||||
|
||||
try {
|
||||
await logger?.info(`Organizing ebook: ${downloadPath}`);
|
||||
|
||||
// Get file info
|
||||
const stats = await fs.stat(downloadPath);
|
||||
if (!stats.isFile()) {
|
||||
throw new Error('Ebook download path must be a file');
|
||||
}
|
||||
|
||||
// Detect format from extension
|
||||
const ext = path.extname(downloadPath).toLowerCase().slice(1);
|
||||
const ebookFormats = ['epub', 'pdf', 'mobi', 'azw', 'azw3', 'fb2', 'cbz', 'cbr'];
|
||||
if (!ebookFormats.includes(ext)) {
|
||||
throw new Error(`Unsupported ebook format: ${ext}`);
|
||||
}
|
||||
|
||||
result.format = ext;
|
||||
await logger?.info(`Detected ebook format: ${ext}`);
|
||||
|
||||
// Build target directory using same template as audiobooks
|
||||
const targetDir = this.buildTargetPath(
|
||||
this.mediaDir,
|
||||
template,
|
||||
metadata.author,
|
||||
metadata.title,
|
||||
undefined, // narrator
|
||||
metadata.asin,
|
||||
metadata.year
|
||||
);
|
||||
|
||||
await logger?.info(`Target directory: ${targetDir}`);
|
||||
|
||||
// Create target directory
|
||||
await fs.mkdir(targetDir, { recursive: true });
|
||||
|
||||
// Build target filename (sanitize source filename)
|
||||
const sourceFilename = path.basename(downloadPath);
|
||||
const targetFilename = this.sanitizePath(sourceFilename);
|
||||
const targetPath = path.join(targetDir, targetFilename);
|
||||
|
||||
// Check if target already exists
|
||||
try {
|
||||
await fs.access(targetPath);
|
||||
await logger?.info(`Ebook already exists at target, skipping copy: ${targetFilename}`);
|
||||
result.success = true;
|
||||
result.targetPath = targetDir;
|
||||
return result;
|
||||
} catch {
|
||||
// File doesn't exist, continue with copy
|
||||
}
|
||||
|
||||
// Copy ebook file (don't delete original in case of direct download retry)
|
||||
await fs.copyFile(downloadPath, targetPath);
|
||||
await fs.chmod(targetPath, 0o644);
|
||||
|
||||
await logger?.info(`Copied ebook: ${targetFilename}`);
|
||||
|
||||
// Clean up source file (for direct HTTP downloads, we don't need to keep the original)
|
||||
try {
|
||||
await fs.unlink(downloadPath);
|
||||
await logger?.info(`Cleaned up source file: ${sourceFilename}`);
|
||||
} catch {
|
||||
// Ignore cleanup errors
|
||||
}
|
||||
|
||||
result.success = true;
|
||||
result.targetPath = targetDir;
|
||||
|
||||
await logger?.info(`Ebook organization complete: ${targetFilename}`);
|
||||
|
||||
return result;
|
||||
} catch (error) {
|
||||
await logger?.error(`Ebook organization failed: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
result.errors.push(error instanceof Error ? error.message : 'Unknown error');
|
||||
return result;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -624,6 +624,161 @@ export class RankingAlgorithm {
|
||||
}
|
||||
}
|
||||
|
||||
// =========================================================================
|
||||
// EBOOK RANKING (simplified algorithm for ebook search results)
|
||||
// =========================================================================
|
||||
|
||||
export interface EbookResult {
|
||||
md5: string;
|
||||
title: string;
|
||||
author: string;
|
||||
format: string; // epub, pdf, mobi, etc.
|
||||
fileSize?: number; // in bytes
|
||||
downloadUrls: string[];
|
||||
source: 'annas_archive' | 'prowlarr'; // Source of the result
|
||||
indexerId?: number; // Prowlarr indexer ID (if applicable)
|
||||
}
|
||||
|
||||
export interface EbookRequest {
|
||||
title: string;
|
||||
author: string;
|
||||
preferredFormat: string; // User's preferred format (epub, pdf, etc.)
|
||||
}
|
||||
|
||||
export interface RankedEbook extends EbookResult {
|
||||
score: number; // Total score (0-100)
|
||||
rank: number;
|
||||
breakdown: {
|
||||
formatScore: number; // 0-40 points
|
||||
sizeScore: number; // 0-30 points (inverted - smaller is better)
|
||||
sourceScore: number; // 0-30 points (Anna's Archive priority)
|
||||
notes: string[];
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Rank ebook search results
|
||||
* Scoring priorities (inverted from audiobooks):
|
||||
* - Format match: 40 points (matching preferred format)
|
||||
* - Size: 30 points (smaller files = better, inverted from audiobooks)
|
||||
* - Source: 30 points (Anna's Archive priority for reliability)
|
||||
*/
|
||||
export function rankEbooks(
|
||||
results: EbookResult[],
|
||||
request: EbookRequest
|
||||
): RankedEbook[] {
|
||||
const preferredFormat = request.preferredFormat.toLowerCase();
|
||||
|
||||
const ranked = results.map((result): RankedEbook => {
|
||||
const notes: string[] = [];
|
||||
|
||||
// ========== FORMAT SCORING (0-40 points) ==========
|
||||
// Exact format match gets full points
|
||||
// Similar formats get partial credit
|
||||
let formatScore = 0;
|
||||
const resultFormat = result.format.toLowerCase();
|
||||
|
||||
if (resultFormat === preferredFormat) {
|
||||
formatScore = 40;
|
||||
notes.push(`✓ Preferred format (${result.format.toUpperCase()})`);
|
||||
} else {
|
||||
// Partial credit for compatible formats
|
||||
const ebookFormatGroups = [
|
||||
['epub', 'kepub'], // EPUB family
|
||||
['mobi', 'azw', 'azw3'], // Kindle family
|
||||
['pdf'], // PDF standalone
|
||||
['fb2', 'fb2.zip'], // FB2 family
|
||||
['cbz', 'cbr'], // Comic formats
|
||||
];
|
||||
|
||||
const preferredGroup = ebookFormatGroups.find(g => g.includes(preferredFormat));
|
||||
const resultGroup = ebookFormatGroups.find(g => g.includes(resultFormat));
|
||||
|
||||
if (preferredGroup && resultGroup && preferredGroup === resultGroup) {
|
||||
formatScore = 30; // Same family
|
||||
notes.push(`Similar format (${result.format.toUpperCase()})`);
|
||||
} else if (resultFormat === 'epub') {
|
||||
formatScore = 25; // EPUB is universally convertible
|
||||
notes.push(`Convertible format (${result.format.toUpperCase()})`);
|
||||
} else if (resultFormat === 'pdf') {
|
||||
formatScore = 15; // PDF is common but less flexible
|
||||
notes.push(`PDF format (less flexible)`);
|
||||
} else {
|
||||
formatScore = 10; // Other formats
|
||||
notes.push(`Different format (${result.format.toUpperCase()})`);
|
||||
}
|
||||
}
|
||||
|
||||
// ========== SIZE SCORING (0-30 points, inverted) ==========
|
||||
// For ebooks, smaller files are generally better (cleaner, no bloat)
|
||||
// Typical ebook sizes: 0.5-5 MB (good), 5-20 MB (has images), 20+ MB (may have issues)
|
||||
let sizeScore = 0;
|
||||
|
||||
if (result.fileSize !== undefined && result.fileSize > 0) {
|
||||
const sizeMB = result.fileSize / (1024 * 1024);
|
||||
|
||||
if (sizeMB <= 2) {
|
||||
sizeScore = 30; // Ideal size
|
||||
notes.push('✓ Optimal file size');
|
||||
} else if (sizeMB <= 5) {
|
||||
sizeScore = 25; // Good size
|
||||
notes.push('Good file size');
|
||||
} else if (sizeMB <= 15) {
|
||||
sizeScore = 20; // Has images, acceptable
|
||||
notes.push('Larger file (may have images)');
|
||||
} else if (sizeMB <= 50) {
|
||||
sizeScore = 10; // Large, possibly bloated
|
||||
notes.push('⚠️ Large file size');
|
||||
} else {
|
||||
sizeScore = 5; // Very large, suspicious
|
||||
notes.push('⚠️ Very large file (may include extras)');
|
||||
}
|
||||
} else {
|
||||
// No size info - give middle score
|
||||
sizeScore = 15;
|
||||
notes.push('File size unknown');
|
||||
}
|
||||
|
||||
// ========== SOURCE SCORING (0-30 points) ==========
|
||||
// Anna's Archive is the primary reliable source
|
||||
// Future: Prowlarr indexers will get configurable priority
|
||||
let sourceScore = 0;
|
||||
|
||||
if (result.source === 'annas_archive') {
|
||||
sourceScore = 30; // Full points for Anna's Archive
|
||||
notes.push('✓ Anna\'s Archive (reliable)');
|
||||
} else if (result.source === 'prowlarr') {
|
||||
// Future: Use indexer priority from config
|
||||
sourceScore = 15; // Base score for Prowlarr results
|
||||
notes.push('Prowlarr indexer');
|
||||
}
|
||||
|
||||
const totalScore = formatScore + sizeScore + sourceScore;
|
||||
|
||||
return {
|
||||
...result,
|
||||
score: totalScore,
|
||||
rank: 0, // Will be assigned after sorting
|
||||
breakdown: {
|
||||
formatScore,
|
||||
sizeScore,
|
||||
sourceScore,
|
||||
notes,
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
// Sort by score descending
|
||||
ranked.sort((a, b) => b.score - a.score);
|
||||
|
||||
// Assign ranks
|
||||
ranked.forEach((r, index) => {
|
||||
r.rank = index + 1;
|
||||
});
|
||||
|
||||
return ranked;
|
||||
}
|
||||
|
||||
// Singleton instance
|
||||
let ranker: RankingAlgorithm | null = null;
|
||||
|
||||
|
||||
@@ -20,6 +20,7 @@ const jobQueueMock = vi.hoisted(() => ({
|
||||
addSearchJob: vi.fn(),
|
||||
addDownloadJob: vi.fn(),
|
||||
addNotificationJob: vi.fn(() => Promise.resolve()),
|
||||
addSearchEbookJob: vi.fn(() => Promise.resolve()),
|
||||
}));
|
||||
const downloadEbookMock = vi.hoisted(() => vi.fn());
|
||||
const fsMock = vi.hoisted(() => ({
|
||||
@@ -355,42 +356,75 @@ describe('Request action routes', () => {
|
||||
expect(payload.error).toMatch(/Cannot fetch e-book/);
|
||||
});
|
||||
|
||||
it('returns 400 when audiobook directory is missing', async () => {
|
||||
it('creates ebook request and triggers search job', async () => {
|
||||
configState.values.set('ebook_sidecar_enabled', 'true');
|
||||
|
||||
// Mock parent request lookup
|
||||
prismaMock.request.findUnique.mockResolvedValueOnce({
|
||||
id: 'req-6',
|
||||
userId: 'user-1',
|
||||
audiobookId: 'ab-1',
|
||||
status: 'downloaded',
|
||||
audiobook: { title: 'Title', author: 'Author', audibleAsin: 'ASIN' },
|
||||
audiobook: { id: 'ab-1', title: 'Title', author: 'Author', audibleAsin: 'ASIN123' },
|
||||
});
|
||||
|
||||
// Mock check for existing ebook request
|
||||
prismaMock.request.findFirst.mockResolvedValueOnce(null);
|
||||
|
||||
// Mock ebook request creation
|
||||
prismaMock.request.create.mockResolvedValueOnce({
|
||||
id: 'ebook-req-1',
|
||||
type: 'ebook',
|
||||
parentRequestId: 'req-6',
|
||||
});
|
||||
fsMock.access.mockRejectedValueOnce(new Error('missing'));
|
||||
|
||||
const { POST } = await import('@/app/api/requests/[id]/fetch-ebook/route');
|
||||
const response = await POST({} as any, { params: Promise.resolve({ id: 'req-6' }) });
|
||||
const payload = await response.json();
|
||||
|
||||
expect(response.status).toBe(400);
|
||||
expect(payload.error).toMatch(/directory not found/);
|
||||
expect(payload.success).toBe(true);
|
||||
expect(payload.message).toMatch(/created/i);
|
||||
expect(payload.requestId).toBe('ebook-req-1');
|
||||
expect(prismaMock.request.create).toHaveBeenCalledWith({
|
||||
data: expect.objectContaining({
|
||||
type: 'ebook',
|
||||
parentRequestId: 'req-6',
|
||||
status: 'pending',
|
||||
}),
|
||||
});
|
||||
expect(jobQueueMock.addSearchEbookJob).toHaveBeenCalledWith(
|
||||
'ebook-req-1',
|
||||
expect.objectContaining({
|
||||
id: 'ab-1',
|
||||
title: 'Title',
|
||||
author: 'Author',
|
||||
asin: 'ASIN123',
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
it('downloads ebook and returns success', async () => {
|
||||
it('retries existing failed ebook request', async () => {
|
||||
configState.values.set('ebook_sidecar_enabled', 'true');
|
||||
configState.values.set('media_dir', '/media/audiobooks');
|
||||
configState.values.set('audiobook_path_template', '{author}/{title} {asin}');
|
||||
configState.values.set('ebook_sidecar_preferred_format', 'epub');
|
||||
configState.values.set('ebook_sidecar_base_url', 'https://ebooks.example');
|
||||
configState.values.set('ebook_sidecar_flaresolverr_url', 'http://flaresolverr');
|
||||
|
||||
// Mock parent request lookup
|
||||
prismaMock.request.findUnique.mockResolvedValueOnce({
|
||||
id: 'req-7',
|
||||
userId: 'user-1',
|
||||
audiobookId: 'ab-1',
|
||||
status: 'available',
|
||||
audiobook: { title: 'Title', author: 'Author', audibleAsin: 'ASIN123' },
|
||||
audiobook: { id: 'ab-1', title: 'Title', author: 'Author', audibleAsin: 'ASIN123' },
|
||||
});
|
||||
prismaMock.audibleCache.findUnique.mockResolvedValueOnce({ releaseDate: '2022-05-01' });
|
||||
fsMock.access.mockResolvedValueOnce(undefined);
|
||||
downloadEbookMock.mockResolvedValueOnce({
|
||||
success: true,
|
||||
format: 'epub',
|
||||
filePath: '/media/audiobooks/Author/Title ASIN123/Title.epub',
|
||||
|
||||
// Mock existing failed ebook request
|
||||
prismaMock.request.findFirst.mockResolvedValueOnce({
|
||||
id: 'ebook-req-existing',
|
||||
status: 'failed',
|
||||
});
|
||||
|
||||
// Mock update for retry
|
||||
prismaMock.request.update.mockResolvedValueOnce({
|
||||
id: 'ebook-req-existing',
|
||||
status: 'pending',
|
||||
});
|
||||
|
||||
const { POST } = await import('@/app/api/requests/[id]/fetch-ebook/route');
|
||||
@@ -398,29 +432,35 @@ describe('Request action routes', () => {
|
||||
const payload = await response.json();
|
||||
|
||||
expect(payload.success).toBe(true);
|
||||
expect(downloadEbookMock).toHaveBeenCalledWith(
|
||||
'ASIN123',
|
||||
'Title',
|
||||
'Author',
|
||||
expect.stringContaining('Title ASIN123'),
|
||||
'epub',
|
||||
'https://ebooks.example',
|
||||
undefined,
|
||||
'http://flaresolverr'
|
||||
);
|
||||
expect(payload.message).toMatch(/retried/i);
|
||||
expect(payload.requestId).toBe('ebook-req-existing');
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'ebook-req-existing' },
|
||||
data: expect.objectContaining({
|
||||
status: 'pending',
|
||||
progress: 0,
|
||||
errorMessage: null,
|
||||
}),
|
||||
});
|
||||
expect(jobQueueMock.addSearchEbookJob).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns failure payload when ebook download fails', async () => {
|
||||
it('returns message when ebook request already exists and in progress', async () => {
|
||||
configState.values.set('ebook_sidecar_enabled', 'true');
|
||||
|
||||
// Mock parent request lookup
|
||||
prismaMock.request.findUnique.mockResolvedValueOnce({
|
||||
id: 'req-8',
|
||||
userId: 'user-1',
|
||||
audiobookId: 'ab-1',
|
||||
status: 'downloaded',
|
||||
audiobook: { title: 'Title', author: 'Author', audibleAsin: 'ASIN123' },
|
||||
audiobook: { id: 'ab-1', title: 'Title', author: 'Author', audibleAsin: 'ASIN123' },
|
||||
});
|
||||
fsMock.access.mockResolvedValueOnce(undefined);
|
||||
downloadEbookMock.mockResolvedValueOnce({
|
||||
success: false,
|
||||
error: 'Download failed',
|
||||
|
||||
// Mock existing in-progress ebook request
|
||||
prismaMock.request.findFirst.mockResolvedValueOnce({
|
||||
id: 'ebook-req-existing',
|
||||
status: 'downloading',
|
||||
});
|
||||
|
||||
const { POST } = await import('@/app/api/requests/[id]/fetch-ebook/route');
|
||||
@@ -428,7 +468,11 @@ describe('Request action routes', () => {
|
||||
const payload = await response.json();
|
||||
|
||||
expect(payload.success).toBe(false);
|
||||
expect(payload.message).toMatch(/Download failed/);
|
||||
expect(payload.message).toMatch(/already exists/i);
|
||||
expect(payload.requestId).toBe('ebook-req-existing');
|
||||
// Should not create new request or trigger search
|
||||
expect(prismaMock.request.create).not.toHaveBeenCalled();
|
||||
expect(jobQueueMock.addSearchEbookJob).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,362 @@
|
||||
/**
|
||||
* Component: Direct Download Processor Tests
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*/
|
||||
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { createPrismaMock } from '../helpers/prisma';
|
||||
|
||||
const prismaMock = createPrismaMock();
|
||||
|
||||
const configServiceMock = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
}));
|
||||
|
||||
const jobQueueMock = vi.hoisted(() => ({
|
||||
addOrganizeJob: vi.fn(() => Promise.resolve()),
|
||||
addMonitorDirectDownloadJob: vi.fn(() => Promise.resolve()),
|
||||
}));
|
||||
|
||||
const ebookScraperMock = vi.hoisted(() => ({
|
||||
extractDownloadUrl: vi.fn(),
|
||||
}));
|
||||
|
||||
const fsMock = vi.hoisted(() => ({
|
||||
mkdir: vi.fn().mockResolvedValue(undefined),
|
||||
stat: vi.fn(),
|
||||
unlink: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
|
||||
const axiosMock = vi.hoisted(() => vi.fn());
|
||||
|
||||
const createWriteStreamMock = vi.hoisted(() => vi.fn());
|
||||
|
||||
vi.mock('@/lib/db', () => ({
|
||||
prisma: prismaMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/services/config.service', () => ({
|
||||
getConfigService: () => configServiceMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/services/job-queue.service', () => ({
|
||||
getJobQueueService: () => jobQueueMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/services/ebook-scraper', () => ebookScraperMock);
|
||||
|
||||
vi.mock('fs/promises', () => ({
|
||||
default: fsMock,
|
||||
...fsMock,
|
||||
}));
|
||||
|
||||
vi.mock('fs', () => ({
|
||||
createWriteStream: createWriteStreamMock,
|
||||
}));
|
||||
|
||||
vi.mock('axios', () => ({
|
||||
default: axiosMock,
|
||||
}));
|
||||
|
||||
describe('processStartDirectDownload', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
configServiceMock.get.mockImplementation(async (key: string) => {
|
||||
if (key === 'downloads_dir') return '/downloads';
|
||||
if (key === 'ebook_sidecar_base_url') return 'https://annas-archive.li';
|
||||
if (key === 'ebook_sidecar_preferred_format') return 'epub';
|
||||
return null;
|
||||
});
|
||||
});
|
||||
|
||||
it('updates request status to downloading', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.findUnique.mockResolvedValue({
|
||||
torrentUrl: JSON.stringify(['https://slow.example.com/book']),
|
||||
});
|
||||
|
||||
// Mock successful download
|
||||
ebookScraperMock.extractDownloadUrl.mockResolvedValue({
|
||||
url: 'https://direct.example.com/book.epub',
|
||||
format: 'epub',
|
||||
});
|
||||
|
||||
// Mock axios stream
|
||||
const mockWriteStream = {
|
||||
on: vi.fn((event, cb) => {
|
||||
if (event === 'finish') setTimeout(cb, 10);
|
||||
return mockWriteStream;
|
||||
}),
|
||||
close: vi.fn(),
|
||||
};
|
||||
createWriteStreamMock.mockReturnValue(mockWriteStream);
|
||||
|
||||
const mockDataStream = {
|
||||
on: vi.fn().mockReturnThis(),
|
||||
pipe: vi.fn().mockReturnValue(mockWriteStream),
|
||||
};
|
||||
axiosMock.mockResolvedValue({
|
||||
data: mockDataStream,
|
||||
headers: { 'content-length': '1000000' },
|
||||
});
|
||||
|
||||
fsMock.stat.mockResolvedValue({ size: 1000000 });
|
||||
prismaMock.request.findUnique.mockResolvedValue({
|
||||
id: 'req-1',
|
||||
audiobookId: 'ab-1',
|
||||
audiobook: { id: 'ab-1' },
|
||||
});
|
||||
|
||||
const { processStartDirectDownload } = await import('@/lib/processors/direct-download.processor');
|
||||
|
||||
const result = await processStartDirectDownload({
|
||||
requestId: 'req-1',
|
||||
downloadHistoryId: 'dh-1',
|
||||
downloadUrl: 'https://slow.example.com/book',
|
||||
targetFilename: 'Test Book.epub',
|
||||
jobId: 'job-1',
|
||||
});
|
||||
|
||||
// Check status updates
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'req-1' },
|
||||
data: expect.objectContaining({
|
||||
status: 'downloading',
|
||||
progress: 0,
|
||||
}),
|
||||
});
|
||||
|
||||
expect(prismaMock.downloadHistory.update).toHaveBeenCalledWith({
|
||||
where: { id: 'dh-1' },
|
||||
data: expect.objectContaining({
|
||||
downloadStatus: 'downloading',
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
it('triggers organize job after successful download', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.findUnique.mockResolvedValue({
|
||||
torrentUrl: JSON.stringify(['https://slow.example.com/book']),
|
||||
});
|
||||
|
||||
ebookScraperMock.extractDownloadUrl.mockResolvedValue({
|
||||
url: 'https://direct.example.com/book.epub',
|
||||
format: 'epub',
|
||||
});
|
||||
|
||||
const mockWriteStream = {
|
||||
on: vi.fn((event, cb) => {
|
||||
if (event === 'finish') setTimeout(cb, 10);
|
||||
return mockWriteStream;
|
||||
}),
|
||||
close: vi.fn(),
|
||||
};
|
||||
createWriteStreamMock.mockReturnValue(mockWriteStream);
|
||||
|
||||
const mockDataStream = {
|
||||
on: vi.fn().mockReturnThis(),
|
||||
pipe: vi.fn().mockReturnValue(mockWriteStream),
|
||||
};
|
||||
axiosMock.mockResolvedValue({
|
||||
data: mockDataStream,
|
||||
headers: { 'content-length': '500000' },
|
||||
});
|
||||
|
||||
fsMock.stat.mockResolvedValue({ size: 500000 });
|
||||
prismaMock.request.findUnique.mockResolvedValue({
|
||||
id: 'req-2',
|
||||
audiobookId: 'ab-2',
|
||||
audiobook: { id: 'ab-2' },
|
||||
});
|
||||
|
||||
const { processStartDirectDownload } = await import('@/lib/processors/direct-download.processor');
|
||||
|
||||
const result = await processStartDirectDownload({
|
||||
requestId: 'req-2',
|
||||
downloadHistoryId: 'dh-2',
|
||||
downloadUrl: 'https://slow.example.com/book2',
|
||||
targetFilename: 'Another Book.epub',
|
||||
jobId: 'job-2',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(jobQueueMock.addOrganizeJob).toHaveBeenCalledWith(
|
||||
'req-2',
|
||||
'ab-2',
|
||||
expect.stringContaining('Another Book.epub')
|
||||
);
|
||||
});
|
||||
|
||||
it('marks request as failed when all download attempts fail', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.findUnique.mockResolvedValue({
|
||||
torrentUrl: JSON.stringify([
|
||||
'https://slow1.example.com/book',
|
||||
'https://slow2.example.com/book',
|
||||
]),
|
||||
});
|
||||
|
||||
// All extract attempts fail
|
||||
ebookScraperMock.extractDownloadUrl.mockResolvedValue(null);
|
||||
|
||||
const { processStartDirectDownload } = await import('@/lib/processors/direct-download.processor');
|
||||
|
||||
const result = await processStartDirectDownload({
|
||||
requestId: 'req-3',
|
||||
downloadHistoryId: 'dh-3',
|
||||
downloadUrl: 'https://slow1.example.com/book',
|
||||
targetFilename: 'Failed Book.epub',
|
||||
jobId: 'job-3',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(false);
|
||||
// Verify the second call (final failure status update)
|
||||
expect(prismaMock.request.update).toHaveBeenLastCalledWith({
|
||||
where: { id: 'req-3' },
|
||||
data: expect.objectContaining({
|
||||
status: 'failed',
|
||||
}),
|
||||
});
|
||||
expect(prismaMock.downloadHistory.update).toHaveBeenLastCalledWith({
|
||||
where: { id: 'dh-3' },
|
||||
data: expect.objectContaining({
|
||||
downloadStatus: 'failed',
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
it('uses FlareSolverr when configured', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.findUnique.mockResolvedValue({
|
||||
torrentUrl: JSON.stringify(['https://slow.example.com/book']),
|
||||
});
|
||||
|
||||
configServiceMock.get.mockImplementation(async (key: string) => {
|
||||
if (key === 'downloads_dir') return '/downloads';
|
||||
if (key === 'ebook_sidecar_base_url') return 'https://annas-archive.li';
|
||||
if (key === 'ebook_sidecar_preferred_format') return 'epub';
|
||||
if (key === 'ebook_sidecar_flaresolverr_url') return 'http://flaresolverr:8191';
|
||||
return null;
|
||||
});
|
||||
|
||||
ebookScraperMock.extractDownloadUrl.mockResolvedValue({
|
||||
url: 'https://direct.example.com/book.epub',
|
||||
format: 'epub',
|
||||
});
|
||||
|
||||
const mockWriteStream = {
|
||||
on: vi.fn((event, cb) => {
|
||||
if (event === 'finish') setTimeout(cb, 10);
|
||||
return mockWriteStream;
|
||||
}),
|
||||
close: vi.fn(),
|
||||
};
|
||||
createWriteStreamMock.mockReturnValue(mockWriteStream);
|
||||
|
||||
const mockDataStream = {
|
||||
on: vi.fn().mockReturnThis(),
|
||||
pipe: vi.fn().mockReturnValue(mockWriteStream),
|
||||
};
|
||||
axiosMock.mockResolvedValue({
|
||||
data: mockDataStream,
|
||||
headers: { 'content-length': '500000' },
|
||||
});
|
||||
|
||||
fsMock.stat.mockResolvedValue({ size: 500000 });
|
||||
prismaMock.request.findUnique.mockResolvedValue({
|
||||
id: 'req-4',
|
||||
audiobookId: 'ab-4',
|
||||
audiobook: { id: 'ab-4' },
|
||||
});
|
||||
|
||||
const { processStartDirectDownload } = await import('@/lib/processors/direct-download.processor');
|
||||
|
||||
await processStartDirectDownload({
|
||||
requestId: 'req-4',
|
||||
downloadHistoryId: 'dh-4',
|
||||
downloadUrl: 'https://slow.example.com/book',
|
||||
targetFilename: 'Flare Book.epub',
|
||||
jobId: 'job-4',
|
||||
});
|
||||
|
||||
expect(ebookScraperMock.extractDownloadUrl).toHaveBeenCalledWith(
|
||||
'https://slow.example.com/book',
|
||||
'https://annas-archive.li',
|
||||
'epub',
|
||||
expect.anything(),
|
||||
'http://flaresolverr:8191'
|
||||
);
|
||||
});
|
||||
|
||||
it('handles errors and updates request status', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.findUnique.mockRejectedValue(new Error('Database error'));
|
||||
|
||||
const { processStartDirectDownload } = await import('@/lib/processors/direct-download.processor');
|
||||
|
||||
await expect(processStartDirectDownload({
|
||||
requestId: 'req-5',
|
||||
downloadHistoryId: 'dh-5',
|
||||
downloadUrl: 'https://slow.example.com/book',
|
||||
targetFilename: 'Error Book.epub',
|
||||
jobId: 'job-5',
|
||||
})).rejects.toThrow('Database error');
|
||||
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'req-5' },
|
||||
data: expect.objectContaining({
|
||||
status: 'failed',
|
||||
errorMessage: 'Database error',
|
||||
}),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('processMonitorDirectDownload', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
it('returns completed status when download file exists', async () => {
|
||||
fsMock.stat.mockResolvedValue({ size: 1000000 });
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
|
||||
const { processMonitorDirectDownload } = await import('@/lib/processors/direct-download.processor');
|
||||
|
||||
const result = await processMonitorDirectDownload({
|
||||
requestId: 'req-m1',
|
||||
downloadHistoryId: 'dh-m1',
|
||||
downloadId: 'dl_unknown',
|
||||
targetPath: '/downloads/book.epub',
|
||||
expectedSize: 1000000,
|
||||
jobId: 'job-m1',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.completed).toBe(true);
|
||||
});
|
||||
|
||||
it('returns not found when download is not tracked', async () => {
|
||||
fsMock.stat.mockRejectedValue(new Error('ENOENT'));
|
||||
|
||||
const { processMonitorDirectDownload } = await import('@/lib/processors/direct-download.processor');
|
||||
|
||||
const result = await processMonitorDirectDownload({
|
||||
requestId: 'req-m2',
|
||||
downloadHistoryId: 'dh-m2',
|
||||
downloadId: 'dl_missing',
|
||||
targetPath: '/downloads/missing.epub',
|
||||
expectedSize: 500000,
|
||||
jobId: 'job-m2',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(false);
|
||||
expect(result.message).toContain('not found');
|
||||
});
|
||||
});
|
||||
@@ -40,6 +40,12 @@ vi.mock('@/lib/services/job-queue.service', () => ({
|
||||
describe('processOrganizeFiles', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
// Default mock for request lookup (processor needs to determine request type)
|
||||
prismaMock.request.findUnique.mockResolvedValue({
|
||||
id: 'req-default',
|
||||
type: 'audiobook', // Default to audiobook type
|
||||
user: { plexUsername: 'testuser' },
|
||||
});
|
||||
});
|
||||
|
||||
it('organizes files and triggers filesystem scan when enabled', async () => {
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
/**
|
||||
* Component: Search Ebook Processor Tests
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*/
|
||||
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { createPrismaMock } from '../helpers/prisma';
|
||||
|
||||
const prismaMock = createPrismaMock();
|
||||
|
||||
const configServiceMock = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
}));
|
||||
|
||||
const jobQueueMock = vi.hoisted(() => ({
|
||||
addStartDirectDownloadJob: vi.fn(() => Promise.resolve()),
|
||||
}));
|
||||
|
||||
const ebookScraperMock = vi.hoisted(() => ({
|
||||
searchByAsin: vi.fn(),
|
||||
searchByTitle: vi.fn(),
|
||||
getSlowDownloadLinks: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/db', () => ({
|
||||
prisma: prismaMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/services/config.service', () => ({
|
||||
getConfigService: () => configServiceMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/services/job-queue.service', () => ({
|
||||
getJobQueueService: () => jobQueueMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/services/ebook-scraper', () => ebookScraperMock);
|
||||
|
||||
describe('processSearchEbook', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
configServiceMock.get.mockImplementation(async (key: string) => {
|
||||
if (key === 'ebook_sidecar_preferred_format') return 'epub';
|
||||
if (key === 'ebook_sidecar_base_url') return 'https://annas-archive.li';
|
||||
return null;
|
||||
});
|
||||
});
|
||||
|
||||
it('searches by ASIN when available and triggers download', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.create.mockResolvedValue({ id: 'dh-1' });
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
|
||||
ebookScraperMock.searchByAsin.mockResolvedValue('abc123md5');
|
||||
ebookScraperMock.getSlowDownloadLinks.mockResolvedValue([
|
||||
'https://slow1.example.com/abc123',
|
||||
'https://slow2.example.com/abc123',
|
||||
]);
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
const result = await processSearchEbook({
|
||||
requestId: 'req-1',
|
||||
audiobook: {
|
||||
id: 'ab-1',
|
||||
title: 'Test Book',
|
||||
author: 'Test Author',
|
||||
asin: 'B001ASIN',
|
||||
},
|
||||
jobId: 'job-1',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.message).toContain('ASIN');
|
||||
expect(ebookScraperMock.searchByAsin).toHaveBeenCalledWith(
|
||||
'B001ASIN',
|
||||
'epub',
|
||||
'https://annas-archive.li',
|
||||
expect.anything(),
|
||||
undefined
|
||||
);
|
||||
expect(jobQueueMock.addStartDirectDownloadJob).toHaveBeenCalledWith(
|
||||
'req-1',
|
||||
'dh-1',
|
||||
'https://slow1.example.com/abc123',
|
||||
'Test Book - Test Author.epub',
|
||||
undefined
|
||||
);
|
||||
});
|
||||
|
||||
it('falls back to title search when ASIN search fails', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.create.mockResolvedValue({ id: 'dh-2' });
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
|
||||
ebookScraperMock.searchByAsin.mockResolvedValue(null);
|
||||
ebookScraperMock.searchByTitle.mockResolvedValue('xyz789md5');
|
||||
ebookScraperMock.getSlowDownloadLinks.mockResolvedValue([
|
||||
'https://slow1.example.com/xyz789',
|
||||
]);
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
const result = await processSearchEbook({
|
||||
requestId: 'req-2',
|
||||
audiobook: {
|
||||
id: 'ab-2',
|
||||
title: 'Another Book',
|
||||
author: 'Another Author',
|
||||
asin: 'B002ASIN',
|
||||
},
|
||||
jobId: 'job-2',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.message).toContain('title search');
|
||||
expect(ebookScraperMock.searchByAsin).toHaveBeenCalled();
|
||||
expect(ebookScraperMock.searchByTitle).toHaveBeenCalledWith(
|
||||
'Another Book',
|
||||
'Another Author',
|
||||
'epub',
|
||||
'https://annas-archive.li',
|
||||
expect.anything(),
|
||||
undefined
|
||||
);
|
||||
});
|
||||
|
||||
it('searches by title when no ASIN is available', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.create.mockResolvedValue({ id: 'dh-3' });
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
|
||||
ebookScraperMock.searchByTitle.mockResolvedValue('noasin123');
|
||||
ebookScraperMock.getSlowDownloadLinks.mockResolvedValue([
|
||||
'https://slow.example.com/noasin123',
|
||||
]);
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
const result = await processSearchEbook({
|
||||
requestId: 'req-3',
|
||||
audiobook: {
|
||||
id: 'ab-3',
|
||||
title: 'No ASIN Book',
|
||||
author: 'No ASIN Author',
|
||||
// No asin field
|
||||
},
|
||||
jobId: 'job-3',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(ebookScraperMock.searchByAsin).not.toHaveBeenCalled();
|
||||
expect(ebookScraperMock.searchByTitle).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('marks request as awaiting_search when no ebook found', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
|
||||
ebookScraperMock.searchByAsin.mockResolvedValue(null);
|
||||
ebookScraperMock.searchByTitle.mockResolvedValue(null);
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
const result = await processSearchEbook({
|
||||
requestId: 'req-4',
|
||||
audiobook: {
|
||||
id: 'ab-4',
|
||||
title: 'Unfindable Book',
|
||||
author: 'Unknown Author',
|
||||
asin: 'B004ASIN',
|
||||
},
|
||||
jobId: 'job-4',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(false);
|
||||
expect(result.message).toContain('re-search');
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'req-4' },
|
||||
data: expect.objectContaining({
|
||||
status: 'awaiting_search',
|
||||
errorMessage: expect.stringContaining('No ebook found'),
|
||||
}),
|
||||
});
|
||||
expect(jobQueueMock.addStartDirectDownloadJob).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('marks request as awaiting_search when no download links available', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
|
||||
ebookScraperMock.searchByAsin.mockResolvedValue('md5nolinks');
|
||||
ebookScraperMock.getSlowDownloadLinks.mockResolvedValue([]);
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
const result = await processSearchEbook({
|
||||
requestId: 'req-5',
|
||||
audiobook: {
|
||||
id: 'ab-5',
|
||||
title: 'Book No Links',
|
||||
author: 'Author No Links',
|
||||
asin: 'B005ASIN',
|
||||
},
|
||||
jobId: 'job-5',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(false);
|
||||
expect(result.message).toContain('re-search');
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'req-5' },
|
||||
data: expect.objectContaining({
|
||||
status: 'awaiting_search',
|
||||
errorMessage: expect.stringContaining('no download links'),
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
it('uses FlareSolverr when configured', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.create.mockResolvedValue({ id: 'dh-6' });
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
|
||||
configServiceMock.get.mockImplementation(async (key: string) => {
|
||||
if (key === 'ebook_sidecar_preferred_format') return 'epub';
|
||||
if (key === 'ebook_sidecar_base_url') return 'https://annas-archive.li';
|
||||
if (key === 'ebook_sidecar_flaresolverr_url') return 'http://flaresolverr:8191';
|
||||
return null;
|
||||
});
|
||||
|
||||
ebookScraperMock.searchByAsin.mockResolvedValue('md5withflare');
|
||||
ebookScraperMock.getSlowDownloadLinks.mockResolvedValue(['https://slow.example.com/flare']);
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
await processSearchEbook({
|
||||
requestId: 'req-6',
|
||||
audiobook: {
|
||||
id: 'ab-6',
|
||||
title: 'Flare Book',
|
||||
author: 'Flare Author',
|
||||
asin: 'B006ASIN',
|
||||
},
|
||||
jobId: 'job-6',
|
||||
});
|
||||
|
||||
expect(ebookScraperMock.searchByAsin).toHaveBeenCalledWith(
|
||||
'B006ASIN',
|
||||
'epub',
|
||||
'https://annas-archive.li',
|
||||
expect.anything(),
|
||||
'http://flaresolverr:8191'
|
||||
);
|
||||
});
|
||||
|
||||
it('fails request on unexpected errors', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
|
||||
ebookScraperMock.searchByAsin.mockRejectedValue(new Error('Network error'));
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
await expect(processSearchEbook({
|
||||
requestId: 'req-7',
|
||||
audiobook: {
|
||||
id: 'ab-7',
|
||||
title: 'Error Book',
|
||||
author: 'Error Author',
|
||||
asin: 'B007ASIN',
|
||||
},
|
||||
jobId: 'job-7',
|
||||
})).rejects.toThrow('Network error');
|
||||
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'req-7' },
|
||||
data: expect.objectContaining({
|
||||
status: 'failed',
|
||||
errorMessage: 'Network error',
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
it('creates download history with correct metadata', async () => {
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.downloadHistory.create.mockResolvedValue({ id: 'dh-8' });
|
||||
prismaMock.downloadHistory.update.mockResolvedValue({});
|
||||
|
||||
ebookScraperMock.searchByAsin.mockResolvedValue('md5metadata');
|
||||
ebookScraperMock.getSlowDownloadLinks.mockResolvedValue([
|
||||
'https://link1.example.com',
|
||||
'https://link2.example.com',
|
||||
]);
|
||||
|
||||
const { processSearchEbook } = await import('@/lib/processors/search-ebook.processor');
|
||||
|
||||
await processSearchEbook({
|
||||
requestId: 'req-8',
|
||||
audiobook: {
|
||||
id: 'ab-8',
|
||||
title: 'Metadata Book',
|
||||
author: 'Metadata Author',
|
||||
asin: 'B008ASIN',
|
||||
},
|
||||
jobId: 'job-8',
|
||||
});
|
||||
|
||||
expect(prismaMock.downloadHistory.create).toHaveBeenCalledWith({
|
||||
data: expect.objectContaining({
|
||||
requestId: 'req-8',
|
||||
indexerName: "Anna's Archive",
|
||||
torrentName: 'Metadata Book - Metadata Author.epub',
|
||||
downloadClient: 'direct',
|
||||
downloadStatus: 'queued',
|
||||
selected: true,
|
||||
qualityScore: 100, // ASIN match = 100
|
||||
}),
|
||||
});
|
||||
|
||||
// Check that all URLs are stored
|
||||
expect(prismaMock.downloadHistory.update).toHaveBeenCalledWith({
|
||||
where: { id: 'dh-8' },
|
||||
data: {
|
||||
torrentUrl: JSON.stringify([
|
||||
'https://link1.example.com',
|
||||
'https://link2.example.com',
|
||||
]),
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -21,6 +21,10 @@ const processorsMock = vi.hoisted(() => ({
|
||||
processRetryMissingTorrents: vi.fn().mockResolvedValue('ok'),
|
||||
processRetryFailedImports: vi.fn().mockResolvedValue('ok'),
|
||||
processCleanupSeededTorrents: vi.fn().mockResolvedValue('ok'),
|
||||
// Ebook processors
|
||||
processSearchEbook: vi.fn().mockResolvedValue('ok'),
|
||||
processStartDirectDownload: vi.fn().mockResolvedValue('ok'),
|
||||
processMonitorDirectDownload: vi.fn().mockResolvedValue('ok'),
|
||||
}));
|
||||
|
||||
const queueMock = vi.hoisted(() => ({
|
||||
@@ -111,6 +115,16 @@ vi.mock('@/lib/processors/cleanup-seeded-torrents.processor', () => ({
|
||||
processCleanupSeededTorrents: processorsMock.processCleanupSeededTorrents,
|
||||
}));
|
||||
|
||||
// Ebook processors
|
||||
vi.mock('@/lib/processors/search-ebook.processor', () => ({
|
||||
processSearchEbook: processorsMock.processSearchEbook,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/processors/direct-download.processor', () => ({
|
||||
processStartDirectDownload: processorsMock.processStartDirectDownload,
|
||||
processMonitorDirectDownload: processorsMock.processMonitorDirectDownload,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/db', () => ({
|
||||
prisma: prismaMock,
|
||||
}));
|
||||
|
||||
@@ -56,6 +56,9 @@ vi.mock('@/lib/utils/file-organizer', () => ({
|
||||
describe('deleteRequest', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
// Default mock for child request queries (audiobook requests check for child ebook requests)
|
||||
prismaMock.request.findMany.mockResolvedValue([]);
|
||||
prismaMock.request.updateMany.mockResolvedValue({ count: 0 });
|
||||
});
|
||||
|
||||
it('returns not found when request is missing', async () => {
|
||||
|
||||
@@ -275,17 +275,8 @@ describe('file organizer', () => {
|
||||
expect(fsMock.copyFile).toHaveBeenCalledWith(sourcePath, targetFile);
|
||||
});
|
||||
|
||||
it('downloads remote cover art and ebook sidecar when enabled', async () => {
|
||||
it('downloads remote cover art when no local cover exists', async () => {
|
||||
configState.values.set('metadata_tagging_enabled', 'false');
|
||||
configState.values.set('ebook_sidecar_enabled', 'true');
|
||||
configState.values.set('ebook_sidecar_preferred_format', 'epub');
|
||||
configState.values.set('ebook_sidecar_base_url', 'https://ebooks.example');
|
||||
configState.values.set('ebook_sidecar_flaresolverr_url', 'http://flaresolverr');
|
||||
|
||||
ebookMock.downloadEbook.mockResolvedValue({
|
||||
success: true,
|
||||
filePath: '/media/Author/Book/book.epub',
|
||||
});
|
||||
|
||||
const organizer = new FileOrganizer('/media', '/tmp');
|
||||
(organizer as any).findAudiobookFiles = vi.fn().mockResolvedValue({
|
||||
@@ -322,18 +313,11 @@ describe('file organizer', () => {
|
||||
'https://images.example/cover.jpg',
|
||||
expect.objectContaining({ responseType: 'arraybuffer' })
|
||||
);
|
||||
expect(ebookMock.downloadEbook).toHaveBeenCalledWith(
|
||||
'ASIN123',
|
||||
'Book',
|
||||
'Author',
|
||||
expectedDir,
|
||||
'epub',
|
||||
'https://ebooks.example',
|
||||
undefined,
|
||||
'http://flaresolverr'
|
||||
);
|
||||
// NOTE: Ebook downloads are now handled as first-class requests through the job queue
|
||||
// The file organizer no longer downloads ebooks inline
|
||||
expect(ebookMock.downloadEbook).not.toHaveBeenCalled();
|
||||
expect(fsMock.copyFile).toHaveBeenCalledWith(sourcePath, targetFile);
|
||||
expect(result.filesMovedCount).toBe(2);
|
||||
expect(result.filesMovedCount).toBe(1);
|
||||
});
|
||||
|
||||
it('records an error when cover art download fails', async () => {
|
||||
@@ -444,36 +428,9 @@ describe('file organizer', () => {
|
||||
expect(result.errors.join(' ')).toContain('Failed to tag 1 file(s) with metadata');
|
||||
});
|
||||
|
||||
it('records ebook sidecar errors when download throws', async () => {
|
||||
configState.values.set('metadata_tagging_enabled', 'false');
|
||||
configState.values.set('ebook_sidecar_enabled', 'true');
|
||||
|
||||
ebookMock.downloadEbook.mockRejectedValue(new Error('ebook down'));
|
||||
|
||||
const organizer = new FileOrganizer('/media', '/tmp');
|
||||
(organizer as any).findAudiobookFiles = vi.fn().mockResolvedValue({
|
||||
audioFiles: ['book.m4b'],
|
||||
coverFile: undefined,
|
||||
isFile: false,
|
||||
});
|
||||
|
||||
const sourcePath = path.join('/downloads', 'book', 'book.m4b');
|
||||
fsMock.access.mockImplementation(async (filePath: string) => {
|
||||
if (path.normalize(filePath) === path.normalize(sourcePath)) return undefined;
|
||||
throw new Error('missing');
|
||||
});
|
||||
fsMock.mkdir.mockResolvedValue(undefined);
|
||||
fsMock.copyFile.mockResolvedValue(undefined);
|
||||
fsMock.chmod.mockResolvedValue(undefined);
|
||||
|
||||
const result = await organizer.organize('/downloads/book', {
|
||||
title: 'Book',
|
||||
author: 'Author',
|
||||
}, '{author}/{title}');
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.errors).toContain('E-book sidecar failed');
|
||||
});
|
||||
// NOTE: The ebook sidecar test was removed because ebook downloads are now
|
||||
// handled as first-class requests through the job queue, not inline during
|
||||
// file organization. See organize-files.processor.ts createEbookRequestIfEnabled().
|
||||
|
||||
it('finds audio files and cover art in nested folders', async () => {
|
||||
const organizer = new FileOrganizer('/media', '/tmp');
|
||||
|
||||
Reference in New Issue
Block a user