mirror of
https://github.com/kikootwo/ReadMeABook.git
synced 2026-09-01 05:08:38 +00:00
Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| bc371860d0 | |||
| ae75c914c9 | |||
| 425129e8c9 | |||
| a013e2191f |
@@ -0,0 +1,26 @@
|
||||
# ReadMeABook Agent Instructions
|
||||
|
||||
These are the repository-wide instructions for all agents. Also follow the project standards and approval workflow in `CLAUDE.md`.
|
||||
|
||||
## Shared Developer Docker Compose
|
||||
|
||||
- The canonical developer Compose file is `C:\GIT\ReadMeABook\docker-compose.yml`.
|
||||
- It is intentionally local-only, marked `skip-worktree`, and contains private machine configuration. Never print its full contents, expose its credentials, or include it in a commit.
|
||||
- The active data bind mounts are hard-coded to `C:\GIT\ReadMeABook\config`, `cache`, `bookdrop`, `pgdata`, and `redis`, so every worktree uses the same developer data. Keep these mounts absolute.
|
||||
- From any ReadMeABook worktree, pass its Git root as `--project-directory`. This keeps `build.context: .` pointed at that worktree's source code while the absolute data mounts continue to use the canonical root directories:
|
||||
|
||||
```powershell
|
||||
$rmabWorktree = (git rev-parse --show-toplevel).Trim()
|
||||
docker compose --project-directory $rmabWorktree -f C:\GIT\ReadMeABook\docker-compose.yml build readmeabook
|
||||
docker compose --project-directory $rmabWorktree -f C:\GIT\ReadMeABook\docker-compose.yml up -d
|
||||
```
|
||||
|
||||
- To build and start in one command:
|
||||
|
||||
```powershell
|
||||
$rmabWorktree = (git rev-parse --show-toplevel).Trim()
|
||||
docker compose --project-directory $rmabWorktree -f C:\GIT\ReadMeABook\docker-compose.yml up -d --build
|
||||
```
|
||||
|
||||
- Do not use a plain `docker compose build readmeabook` unless the active worktree's Compose file has first been verified to contain a `build` section. The tracked production Compose normally references a prebuilt registry image, making that command a no-op.
|
||||
- The developer Compose uses the fixed container name `readmeabook-test`. Before starting it, inspect any existing container with that name. Do not remove or replace an existing container without user approval.
|
||||
@@ -88,8 +88,10 @@ model Request {
|
||||
```
|
||||
|
||||
3. **Delete Media Files**
|
||||
- Path: `[media_dir]/[author]/[title]/`
|
||||
- Path: persisted `Audiobook.filePath` recorded during file organization
|
||||
- Never rebuilds the path from the current `media_dir` or path template
|
||||
- **ONLY deletes title folder** (not author folder)
|
||||
- Missing stored path: skips file deletion and returns `filesDeleted: false`
|
||||
- Handles missing folders gracefully
|
||||
|
||||
4. **Delete from Library Backend**
|
||||
@@ -214,6 +216,8 @@ where: {
|
||||
13. ✅ **Plex deletion not enabled in settings** - Log error, continue with soft delete
|
||||
14. ✅ **Title mismatch in plex_library** - ASIN-based deletion handles title variations (e.g., "(Unabridged)" suffix)
|
||||
15. ✅ **No ASIN available** - Falls back to exact title/author matching
|
||||
16. ✅ **Path template changed after organization** - Deletes only the persisted organized path
|
||||
17. ✅ **Legacy row has no stored path** - Skips file deletion without guessing a path
|
||||
|
||||
## Fixed Issues ✅
|
||||
|
||||
@@ -224,6 +228,12 @@ where: {
|
||||
- **Fix:** Changed plex_library deletion to use ASIN-based matching (same as availability check)
|
||||
- **Result:** Books immediately show as NOT available after deletion, can be re-requested right away
|
||||
|
||||
**2. Changed Path Template Deletes an Unrelated Directory**
|
||||
- **Issue:** Deletion rebuilt a directory from the current media path and template
|
||||
- **Cause:** `Audiobook.filePath` was persisted during organization but omitted from the deletion query
|
||||
- **Fix:** Delete only the persisted path; legacy rows without one skip file cleanup
|
||||
- **Result:** Configuration changes cannot redirect deletion to a newly rendered directory
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
@@ -250,7 +260,6 @@ Queries Updated (deletedAt: null filters):
|
||||
|
||||
**No new config required** - uses existing:
|
||||
- `prowlarr_indexers` (seeding time per indexer)
|
||||
- `media_dir` (file deletion path)
|
||||
|
||||
## Security
|
||||
|
||||
@@ -258,6 +267,7 @@ Queries Updated (deletedAt: null filters):
|
||||
- **Audit Trail:** `deletedBy` tracks admin user ID
|
||||
- **Soft Delete:** Preserves history, prevents permanent data loss
|
||||
- **Confirmation Required:** Prevents accidental deletion
|
||||
- **Stored Path Only:** File cleanup targets the path recorded by the organizer; no template-based fallback
|
||||
|
||||
## Monitoring & Logging
|
||||
|
||||
|
||||
@@ -33,6 +33,14 @@ Manages background job queue using Bull (Redis-backed) for async tasks: searchin
|
||||
5. **plex_recently_added_check** - Lightweight polling of recently added items, match all non-terminal requests
|
||||
6. **match_plex** - Fuzzy match to Plex item (deprecated - now handled by scan_plex)
|
||||
|
||||
## Download Grab Fallback
|
||||
|
||||
- Automatic audiobook search passes every above-threshold release to `download_torrent` in ranked order.
|
||||
- HTTP `429` and `5xx` responses while fetching a `.torrent`/`.nzb` from Prowlarr are source-grab failures, not download-client failures.
|
||||
- `download_torrent` immediately tries the next ranked release; the successful candidate is the only one written to download history.
|
||||
- If every ranked grab is temporarily unavailable, the job remains retryable under Bull's exponential backoff. The global failed handler marks the request failed only after all job attempts are exhausted.
|
||||
- Manual selections carry no alternates and retain their explicit-release behavior.
|
||||
|
||||
## Special Behaviors
|
||||
|
||||
**monitor_download:**
|
||||
|
||||
@@ -19,11 +19,11 @@ Manages recurring/scheduled jobs providing automated tasks (Plex scans, Audible
|
||||
1. **plex_library_scan** - Default: every 6 hours, full library scan, disabled by default (enable after setup)
|
||||
2. **plex_recently_added_check** - Default: every 5 minutes, lightweight polling of top 10 recently added items, enabled by default
|
||||
3. **audible_refresh** - Default: daily midnight, fetches 200 popular + 200 new releases, stores with rankings, disabled by default
|
||||
4. **retry_missing_torrents** - Default: daily midnight, processes union of `awaiting_search` ∪ `awaiting_release` (limit 50), handles both audiobook and ebook requests. Bidirectional transitions: `awaiting_search` → `awaiting_release` when release date is future + `indexer.skip_unreleased` ON; `awaiting_release` → `awaiting_search` + run search when release date has passed or setting OFF. Sole owner of these transitions. Enabled by default.
|
||||
5. **retry_failed_imports** - Default: every 6 hours, re-attempts 'awaiting_import' status (limit 50), enabled by default
|
||||
4. **retry_missing_torrents** - Default: daily midnight, processes union of `awaiting_search` ∪ eligible `awaiting_release` (limit 50), handles both audiobook and ebook requests. Fair ordering: never-searched first, then least-recently searched; future `awaiting_release` rows do not consume retry slots. Bidirectional transitions: `awaiting_search` → `awaiting_release` when release date is future + `indexer.skip_unreleased` ON; `awaiting_release` → `awaiting_search` + run search when release date has passed or setting OFF. Sole owner of these transitions. Enabled by default.
|
||||
5. **retry_failed_imports** - Default: every 6 hours, re-attempts `awaiting_import` status (limit 50), ordered by oldest `lastImportAt` so the backlog rotates fairly; enabled by default
|
||||
6. **find_missing_ebooks** - Default: daily midnight, scans `downloaded` ∪ `available` audiobook requests (limit 50) for missing ebook companions and triggers the existing ebook fetch flow (`addSearchEbookJob`). Gated by `ebook_auto_grab_enabled` AND at least one ebook source enabled (`ebook_annas_archive_enabled` or `ebook_indexer_search_enabled`; legacy `ebook_sidecar_enabled` accepted as Anna's fallback). Skips ebook children in-flight (`pending`, `awaiting_approval`, `searching`, `downloading`, `processing`, `awaiting_search`, `awaiting_release`) or `cancelled`. Retries `failed`/`warn` children up to **5 lifetime auto-retries** per audiobook, tracked in `Request.ebookAutoRetryCount` (nullable; processor-private — manual "Fetch Ebook" never reads/writes it). Per-candidate writes are wrapped in `prisma.$transaction` for race-safety with concurrent auto-grab; counter rolls back if `addSearchEbookJob` throws. Enabled by default. Returns `{ scanned, gapsFound, triggered, created, retried, skippedInFlight, skippedCancelled, skippedCapHit }`.
|
||||
7. **cleanup_seeded_torrents** - Default: every 30 mins, deletes torrents after seeding requirements met. Respects per-indexer `seedingTimeMinutes` AND `ratioLimit` (BOTH required when set; `0` disables that criterion; both `0` = never cleaned up). Undefined ratio with `ratioLimit > 0` = not met (safe-deny). Enabled by default.
|
||||
8. **monitor_rss_feeds** - Default: every 15 mins, checks RSS feeds from enabled indexers, matches against `awaiting_search` requests (audiobook and ebook, limit 100). Query is unchanged — release-date gate is applied AFTER a match is found: if matched book is unreleased + `indexer.skip_unreleased` ON, the match is skipped and request status is NOT mutated (retry job owns transitions). Enabled by default.
|
||||
8. **monitor_rss_feeds** - Default: every 15 mins, checks RSS feeds from enabled indexers, matches against all `awaiting_search` requests (audiobook and ebook) in deterministic pages of 100. Release-date gate is applied AFTER a match is found: if matched book is unreleased + `indexer.skip_unreleased` ON, the match is skipped and request status is NOT mutated (retry job owns transitions). Enabled by default.
|
||||
|
||||
## Architecture: Bull + Cron
|
||||
|
||||
|
||||
@@ -20,6 +20,8 @@ Request → search_indexers → rank_results → download_torrent
|
||||
5. **process_audiobook** - Organize files to media directory
|
||||
6. **update_plex** - Trigger scan, fuzzy match
|
||||
|
||||
Automatic searches keep the ranked candidate list. If Prowlarr returns HTTP `429` or `5xx` while grabbing the selected release, `download_torrent` attributes the error to the indexer/source and falls through to the next release. When all candidates have temporary grab failures, Bull retries the job with backoff.
|
||||
|
||||
## Integration Points
|
||||
|
||||
**Indexers:** Prowlarr (primary), Jackett (fallback)
|
||||
|
||||
@@ -27,7 +27,8 @@ Indexer aggregator for searching multiple torrent/usenet indexers simultaneously
|
||||
- Minimum score threshold: 50/100
|
||||
- Filters applied after ranking to remove poor matches
|
||||
- Ensures at least basic title/author match quality
|
||||
- maxResults: 100 (increased from 50 for broader search)
|
||||
- Audiobook searches rank the full deduplicated result set, then retain the best 100
|
||||
- Interactive responses expose `truncated` and the pre-ranking `rawCount`
|
||||
|
||||
**Example:** "Season of Storms" → finds all "Season of Storms" torrents → ranks by author match → filters score < 50
|
||||
|
||||
@@ -66,16 +67,17 @@ interface TorrentResult {
|
||||
|
||||
**Manual Search** (`POST /api/requests/{id}/manual-search`)
|
||||
- Triggers automatic search job for requests with status: pending, failed, awaiting_search
|
||||
- Searches only enabled indexers (title only, maxResults: 100)
|
||||
- Searches only enabled indexers (title only)
|
||||
- Ranks all results, filters scores < 50
|
||||
- Retains the best 100 qualifying results after ranking
|
||||
- Selects best torrent from filtered results
|
||||
- Updates request status to 'pending'
|
||||
|
||||
**Interactive Search** (`POST /api/requests/{id}/interactive-search`)
|
||||
- Returns ranked torrent results for user selection
|
||||
- Searches only enabled indexers (title only or custom, maxResults: 100)
|
||||
- Searches only enabled indexers (title only or custom)
|
||||
- Accepts optional custom search title in request body
|
||||
- Ranks all results, filters scores < 50
|
||||
- Ranks all results, then returns the best 100 with truncation metadata
|
||||
- Shows table with: rank, title, size, quality score, seeders, indexer, publish date
|
||||
- Editable title field allows search refinement
|
||||
- Available for same statuses as manual search
|
||||
|
||||
@@ -247,7 +247,13 @@ type TorrentState =
|
||||
- `pausedUP` → `seeding` (unified) / `completed` (legacy) — triggers completion in monitor
|
||||
- `stoppedUP` → `seeding` (unified) / `completed` (legacy) — same fix for qBittorrent v5.x
|
||||
- `pausedDL`/`stoppedDL` remain `paused` — download phase genuinely paused
|
||||
- Key insight: any `*UP` state is post-download; any `*DL` state is pre-completion
|
||||
- Key insight: any `*UP` state is post-download; any `*DL` state is pre-completion
|
||||
|
||||
**17. Prowlarr grab errors blamed on qBittorrent** - Source HTTP errors were collapsed into `Failed to add torrent to qBittorrent`, so a rate-limited indexer permanently failed the request. Fixed by:
|
||||
- Forwarding Prowlarr source headers when fetching `.torrent` files
|
||||
- Preserving the upstream HTTP status as a structured source-grab error
|
||||
- Trying the remaining ranked releases on HTTP `429`/`5xx`
|
||||
- Reporting the indexer, release, and upstream status instead of blaming a healthy qBittorrent instance
|
||||
|
||||
## Tech Stack
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ import { z } from 'zod';
|
||||
import { RMABLogger } from '@/lib/utils/logger';
|
||||
|
||||
const logger = RMABLogger.create('API.AudiobookSearch');
|
||||
const MAX_RANKED_RESULTS = 100;
|
||||
|
||||
const SearchSchema = z.object({
|
||||
title: z.string(),
|
||||
@@ -98,7 +99,6 @@ export async function POST(request: NextRequest) {
|
||||
const groupResults = await prowlarr.searchWithVariations(title, author, {
|
||||
categories: group.categories,
|
||||
indexerIds: group.indexerIds,
|
||||
maxResults: 100, // Limit per group
|
||||
});
|
||||
|
||||
logger.debug(`Group ${i + 1} returned ${groupResults.length} results`);
|
||||
@@ -116,6 +116,8 @@ export async function POST(request: NextRequest) {
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
results: [],
|
||||
truncated: false,
|
||||
rawCount: 0,
|
||||
message: 'No torrents/nzbs found',
|
||||
});
|
||||
}
|
||||
@@ -167,6 +169,14 @@ export async function POST(request: NextRequest) {
|
||||
// User can see scores and make their own decision
|
||||
logger.debug(`Ranked ${rankedResults.length} results (no threshold filter - user decides)`);
|
||||
|
||||
const truncated = rankedResults.length > MAX_RANKED_RESULTS;
|
||||
const limitedRankedResults = rankedResults.slice(0, MAX_RANKED_RESULTS);
|
||||
if (truncated) {
|
||||
logger.info(`Audiobook search truncated after ranking: returning top ${MAX_RANKED_RESULTS} of ${rankedResults.length} ranked results`, {
|
||||
rawCount: results.length,
|
||||
});
|
||||
}
|
||||
|
||||
// Log top 3 results with detailed score breakdown for debugging
|
||||
const top3 = rankedResults.slice(0, 3);
|
||||
if (top3.length > 0) {
|
||||
@@ -198,7 +208,7 @@ export async function POST(request: NextRequest) {
|
||||
}
|
||||
|
||||
// Add rank position to each result
|
||||
const resultsWithRank = rankedResults.map((result, index) => ({
|
||||
const resultsWithRank = limitedRankedResults.map((result, index) => ({
|
||||
...result,
|
||||
rank: index + 1,
|
||||
}));
|
||||
@@ -206,8 +216,10 @@ export async function POST(request: NextRequest) {
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
results: resultsWithRank,
|
||||
message: rankedResults.length > 0
|
||||
? `Found ${rankedResults.length} results`
|
||||
truncated,
|
||||
rawCount: results.length,
|
||||
message: limitedRankedResults.length > 0
|
||||
? `Found ${limitedRankedResults.length} results`
|
||||
: 'No results found',
|
||||
});
|
||||
} catch (error) {
|
||||
|
||||
@@ -15,6 +15,7 @@ import { RMABLogger } from '@/lib/utils/logger';
|
||||
import { resolveInteractiveSearchAccess } from '@/lib/utils/permissions';
|
||||
|
||||
const logger = RMABLogger.create('API.InteractiveSearch');
|
||||
const MAX_RANKED_RESULTS = 100;
|
||||
|
||||
/**
|
||||
* POST /api/requests/[id]/interactive-search
|
||||
@@ -151,7 +152,6 @@ export async function POST(
|
||||
const groupResults = await prowlarr.searchWithVariations(searchTitle, searchAuthor, {
|
||||
categories: group.categories,
|
||||
indexerIds: group.indexerIds,
|
||||
maxResults: 100,
|
||||
});
|
||||
|
||||
logger.debug(`Group ${i + 1} returned ${groupResults.length} results`);
|
||||
@@ -169,6 +169,8 @@ export async function POST(
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
results: [],
|
||||
truncated: false,
|
||||
rawCount: 0,
|
||||
message: 'No torrents/nzbs found',
|
||||
});
|
||||
}
|
||||
@@ -214,6 +216,14 @@ export async function POST(
|
||||
// User can see scores and make their own decision
|
||||
logger.debug(`Ranked ${rankedResults.length} results (no threshold filter - user decides)`);
|
||||
|
||||
const truncated = rankedResults.length > MAX_RANKED_RESULTS;
|
||||
const limitedRankedResults = rankedResults.slice(0, MAX_RANKED_RESULTS);
|
||||
if (truncated) {
|
||||
logger.info(`Interactive search truncated after ranking: returning top ${MAX_RANKED_RESULTS} of ${rankedResults.length} ranked results`, {
|
||||
rawCount: results.length,
|
||||
});
|
||||
}
|
||||
|
||||
// Log top 3 results with detailed score breakdown for debugging
|
||||
const top3 = rankedResults.slice(0, 3);
|
||||
if (top3.length > 0) {
|
||||
@@ -245,7 +255,7 @@ export async function POST(
|
||||
}
|
||||
|
||||
// Add rank position to each result
|
||||
const resultsWithRank = rankedResults.map((result, index) => ({
|
||||
const resultsWithRank = limitedRankedResults.map((result, index) => ({
|
||||
...result,
|
||||
rank: index + 1,
|
||||
}));
|
||||
@@ -253,8 +263,10 @@ export async function POST(
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
results: resultsWithRank,
|
||||
message: rankedResults.length > 0
|
||||
? `Found ${rankedResults.length} results`
|
||||
truncated,
|
||||
rawCount: results.length,
|
||||
message: limitedRankedResults.length > 0
|
||||
? `Found ${limitedRankedResults.length} results`
|
||||
: 'No results found',
|
||||
});
|
||||
} catch (error) {
|
||||
|
||||
@@ -14,6 +14,7 @@ import { PathMapper, PathMappingConfig } from '../utils/path-mapper';
|
||||
import {
|
||||
IDownloadClient, DownloadClientType, ProtocolType,
|
||||
DownloadInfo, DownloadStatus, AddDownloadOptions, ConnectionTestResult,
|
||||
DownloadSourceError,
|
||||
} from '../interfaces/download-client.interface';
|
||||
|
||||
const parseTorrent = (parseTorrentModule as any).default || parseTorrentModule;
|
||||
@@ -193,7 +194,7 @@ export class DelugeService implements IDownloadClient {
|
||||
torrentResponse = await axios.get(torrentUrl, {
|
||||
responseType: 'arraybuffer', maxRedirects: 0,
|
||||
validateStatus: (s) => s >= 200 && s < 300, timeout: DOWNLOAD_CLIENT_TIMEOUT,
|
||||
headers: { 'User-Agent': RMAB_USER_AGENT },
|
||||
headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders },
|
||||
});
|
||||
if (torrentResponse.data.length > 0) {
|
||||
const magnetMatch = torrentResponse.data.toString().match(/^magnet:\?[^\s]+$/);
|
||||
@@ -206,10 +207,27 @@ export class DelugeService implements IDownloadClient {
|
||||
const loc = error.response.headers['location'];
|
||||
if (loc?.startsWith('magnet:')) return this.addMagnetLink(loc, category, options);
|
||||
if (loc?.startsWith('http://') || loc?.startsWith('https://')) {
|
||||
try { torrentResponse = await axios.get(loc, { responseType: 'arraybuffer', timeout: DOWNLOAD_CLIENT_TIMEOUT, maxRedirects: 5, headers: { 'User-Agent': RMAB_USER_AGENT } }); }
|
||||
catch { throw new Error('Failed to download torrent file after redirect'); }
|
||||
try { torrentResponse = await axios.get(loc, { responseType: 'arraybuffer', timeout: DOWNLOAD_CLIENT_TIMEOUT, maxRedirects: 5, headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders } }); }
|
||||
catch (redirectError) {
|
||||
if (axios.isAxiosError(redirectError) && redirectError.response?.status) {
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${redirectError.response.status}`,
|
||||
redirectError.response.status,
|
||||
loc,
|
||||
redirectError
|
||||
);
|
||||
}
|
||||
throw new Error('Failed to download torrent file after redirect');
|
||||
}
|
||||
} else { throw new Error(`Invalid redirect location: ${loc}`); }
|
||||
} else { throw new Error(`Failed to download torrent: HTTP ${status}`); }
|
||||
} else {
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${status}`,
|
||||
status,
|
||||
torrentUrl,
|
||||
error
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const torrentBuffer = Buffer.from(torrentResponse.data);
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
DownloadStatus,
|
||||
AddDownloadOptions,
|
||||
ConnectionTestResult,
|
||||
DownloadSourceError,
|
||||
} from '../interfaces/download-client.interface';
|
||||
|
||||
const logger = RMABLogger.create('NZBGet');
|
||||
@@ -255,7 +256,12 @@ export class NZBGetService implements IDownloadClient {
|
||||
if (axios.isAxiosError(error)) {
|
||||
const status = error.response?.status;
|
||||
if (status) {
|
||||
throw new Error(`Failed to download NZB file: HTTP ${status} from source URL`);
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${status}`,
|
||||
status,
|
||||
url,
|
||||
error
|
||||
);
|
||||
}
|
||||
if (error.code === 'ECONNREFUSED') {
|
||||
throw new Error('Failed to download NZB file: Connection refused. Is Prowlarr running?');
|
||||
|
||||
@@ -20,6 +20,7 @@ import {
|
||||
DownloadStatus,
|
||||
AddDownloadOptions,
|
||||
ConnectionTestResult,
|
||||
DownloadSourceError,
|
||||
} from '../interfaces/download-client.interface';
|
||||
|
||||
// Handle both ESM and CommonJS imports
|
||||
@@ -33,6 +34,7 @@ export interface AddTorrentOptions {
|
||||
tags?: string[];
|
||||
paused?: boolean;
|
||||
skipChecking?: boolean;
|
||||
sourceHeaders?: Record<string, string>;
|
||||
}
|
||||
|
||||
export interface TorrentInfo {
|
||||
@@ -282,6 +284,11 @@ export class QBittorrentService implements IDownloadClient {
|
||||
return await this.addTorrentFile(url, category, options);
|
||||
}
|
||||
} catch (error) {
|
||||
// Preserve source/indexer failures so the processor can try another release.
|
||||
if (error instanceof DownloadSourceError) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
// Try re-authenticating once if we get a 403 — only meaningful when credentials are configured.
|
||||
// In auth-optional mode a 403 means the server actually wants auth (e.g. IP no longer whitelisted),
|
||||
// so retrying login is pointless and would mask the real error.
|
||||
@@ -378,6 +385,10 @@ export class QBittorrentService implements IDownloadClient {
|
||||
maxRedirects: 0,
|
||||
validateStatus: (status) => status >= 200 && status < 300, // Only 2xx is success
|
||||
timeout: DOWNLOAD_CLIENT_TIMEOUT,
|
||||
headers: {
|
||||
'User-Agent': RMAB_USER_AGENT,
|
||||
...options?.sourceHeaders,
|
||||
},
|
||||
});
|
||||
|
||||
logger.info(` Got 2xx response, size=${torrentResponse.data.length} bytes`);
|
||||
@@ -421,10 +432,22 @@ export class QBittorrentService implements IDownloadClient {
|
||||
responseType: 'arraybuffer',
|
||||
timeout: DOWNLOAD_CLIENT_TIMEOUT,
|
||||
maxRedirects: 5,
|
||||
headers: {
|
||||
'User-Agent': RMAB_USER_AGENT,
|
||||
...options?.sourceHeaders,
|
||||
},
|
||||
});
|
||||
logger.info(` After following redirect: size=${torrentResponse.data.length} bytes`);
|
||||
} catch (redirectError) {
|
||||
logger.error('Failed to follow redirect', { error: redirectError instanceof Error ? redirectError.message : String(redirectError) });
|
||||
if (axios.isAxiosError(redirectError) && redirectError.response?.status) {
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${redirectError.response.status}`,
|
||||
redirectError.response.status,
|
||||
location,
|
||||
redirectError
|
||||
);
|
||||
}
|
||||
throw new Error('Failed to download torrent file after redirect');
|
||||
}
|
||||
} else {
|
||||
@@ -433,7 +456,12 @@ export class QBittorrentService implements IDownloadClient {
|
||||
} else {
|
||||
// Non-redirect error (4xx, 5xx)
|
||||
logger.error(`HTTP error ${status}`, { error: error.message });
|
||||
throw new Error(`Failed to download torrent: HTTP ${status}`);
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${status}`,
|
||||
status,
|
||||
torrentUrl,
|
||||
error
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1077,6 +1105,7 @@ export class QBittorrentService implements IDownloadClient {
|
||||
category: options?.category,
|
||||
paused: options?.paused,
|
||||
tags: ['audiobook'],
|
||||
sourceHeaders: options?.sourceHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
DownloadStatus,
|
||||
AddDownloadOptions,
|
||||
ConnectionTestResult,
|
||||
DownloadSourceError,
|
||||
} from '../interfaces/download-client.interface';
|
||||
|
||||
const logger = RMABLogger.create('SABnzbd');
|
||||
@@ -519,7 +520,12 @@ export class SABnzbdService implements IDownloadClient {
|
||||
if (axios.isAxiosError(error)) {
|
||||
const status = error.response?.status;
|
||||
if (status) {
|
||||
throw new Error(`Failed to download NZB file: HTTP ${status} from source URL`);
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${status}`,
|
||||
status,
|
||||
url,
|
||||
error
|
||||
);
|
||||
}
|
||||
if (error.code === 'ECONNREFUSED') {
|
||||
throw new Error('Failed to download NZB file: Connection refused. Is Prowlarr running?');
|
||||
|
||||
@@ -19,6 +19,7 @@ import {
|
||||
DownloadStatus,
|
||||
AddDownloadOptions,
|
||||
ConnectionTestResult,
|
||||
DownloadSourceError,
|
||||
} from '../interfaces/download-client.interface';
|
||||
|
||||
// Handle both ESM and CommonJS imports
|
||||
@@ -278,7 +279,7 @@ export class TransmissionService implements IDownloadClient {
|
||||
maxRedirects: 0,
|
||||
validateStatus: (status) => status >= 200 && status < 300,
|
||||
timeout: DOWNLOAD_CLIENT_TIMEOUT,
|
||||
headers: { 'User-Agent': RMAB_USER_AGENT },
|
||||
headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders },
|
||||
});
|
||||
|
||||
// Check if response body is a magnet link
|
||||
@@ -308,16 +309,29 @@ export class TransmissionService implements IDownloadClient {
|
||||
responseType: 'arraybuffer',
|
||||
timeout: DOWNLOAD_CLIENT_TIMEOUT,
|
||||
maxRedirects: 5,
|
||||
headers: { 'User-Agent': RMAB_USER_AGENT },
|
||||
headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders },
|
||||
});
|
||||
} catch {
|
||||
} catch (redirectError) {
|
||||
if (axios.isAxiosError(redirectError) && redirectError.response?.status) {
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${redirectError.response.status}`,
|
||||
redirectError.response.status,
|
||||
location,
|
||||
redirectError
|
||||
);
|
||||
}
|
||||
throw new Error('Failed to download torrent file after redirect');
|
||||
}
|
||||
} else {
|
||||
throw new Error(`Invalid redirect location: ${location}`);
|
||||
}
|
||||
} else {
|
||||
throw new Error(`Failed to download torrent: HTTP ${status}`);
|
||||
throw new DownloadSourceError(
|
||||
`Grab failed: source returned HTTP ${status}`,
|
||||
status,
|
||||
torrentUrl,
|
||||
error
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -106,6 +106,33 @@ export interface AddDownloadOptions {
|
||||
sourceHeaders?: Record<string, string>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Failure while fetching a .torrent or .nzb from its source URL.
|
||||
* Keeping this distinct from download-client API failures lets the pipeline
|
||||
* try another release without blaming a healthy download client.
|
||||
*/
|
||||
export class DownloadSourceError extends Error {
|
||||
readonly name = 'DownloadSourceError';
|
||||
|
||||
constructor(
|
||||
message: string,
|
||||
public readonly status: number | undefined,
|
||||
public readonly sourceUrl: string,
|
||||
public readonly originalError?: unknown
|
||||
) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
/** HTTP source failures that may clear on their own or succeed via another indexer. */
|
||||
export function isRetryableDownloadSourceError(
|
||||
error: unknown
|
||||
): error is DownloadSourceError {
|
||||
return error instanceof DownloadSourceError
|
||||
&& typeof error.status === 'number'
|
||||
&& (error.status === 429 || (error.status >= 500 && error.status < 600));
|
||||
}
|
||||
|
||||
/** Result of a connection test */
|
||||
export interface ConnectionTestResult {
|
||||
success: boolean;
|
||||
|
||||
@@ -10,6 +10,10 @@ import { getDownloadClientManager } from '../services/download-client-manager.se
|
||||
import { ProwlarrService } from '../integrations/prowlarr.service';
|
||||
import { RMABLogger } from '../utils/logger';
|
||||
import { isTransientConnectionError } from '../utils/connection-errors';
|
||||
import {
|
||||
DownloadSourceError,
|
||||
isRetryableDownloadSourceError,
|
||||
} from '../interfaces/download-client.interface';
|
||||
|
||||
/**
|
||||
* Process download job
|
||||
@@ -17,7 +21,8 @@ import { isTransientConnectionError } from '../utils/connection-errors';
|
||||
* Adds selected result to download client and starts monitoring
|
||||
*/
|
||||
export async function processDownloadTorrent(payload: DownloadTorrentPayload): Promise<any> {
|
||||
const { requestId, audiobook, torrent, jobId } = payload;
|
||||
const { requestId, audiobook, torrent, alternateTorrents = [], jobId } = payload;
|
||||
const candidates = [torrent, ...alternateTorrents];
|
||||
|
||||
const logger = RMABLogger.forJob(jobId, 'DownloadTorrent');
|
||||
|
||||
@@ -27,6 +32,7 @@ export async function processDownloadTorrent(payload: DownloadTorrentPayload): P
|
||||
seeders: torrent.seeders,
|
||||
format: torrent.format,
|
||||
indexer: torrent.indexer,
|
||||
alternateCount: alternateTorrents.length,
|
||||
});
|
||||
|
||||
try {
|
||||
@@ -43,24 +49,9 @@ export async function processDownloadTorrent(payload: DownloadTorrentPayload): P
|
||||
},
|
||||
});
|
||||
|
||||
// Detect protocol from result and get appropriate client
|
||||
const isUsenet = ProwlarrService.isNZBResult(torrent);
|
||||
const protocol = isUsenet ? 'usenet' : 'torrent';
|
||||
const config = await getConfigService();
|
||||
const manager = getDownloadClientManager(config);
|
||||
|
||||
const client = await manager.getClientServiceForProtocol(protocol);
|
||||
|
||||
if (!client) {
|
||||
throw new Error(`No ${protocol} download client configured. Please add a ${protocol} client in Settings > Download Clients.`);
|
||||
}
|
||||
|
||||
// Get client config for category
|
||||
const clientConfig = await manager.getClientForProtocol(protocol);
|
||||
const category = clientConfig?.category || 'readmeabook';
|
||||
|
||||
logger.info(`Routing to ${client.clientType} (${client.protocol})`);
|
||||
|
||||
// Include Prowlarr API key as source header so NZB/torrent downloads from
|
||||
// Prowlarr proxy URLs are authenticated (fixes 403 for indexers like NZBFinder)
|
||||
const prowlarrApiKey = (await config.getMany(['prowlarr_api_key'])).prowlarr_api_key || process.env.PROWLARR_API_KEY;
|
||||
@@ -69,90 +60,149 @@ export async function processDownloadTorrent(payload: DownloadTorrentPayload): P
|
||||
sourceHeaders['X-Api-Key'] = prowlarrApiKey;
|
||||
}
|
||||
|
||||
// Add download via unified interface
|
||||
const downloadClientId = await client.addDownload(torrent.downloadUrl, {
|
||||
category,
|
||||
priority: 'normal',
|
||||
sourceHeaders,
|
||||
});
|
||||
for (let candidateIndex = 0; candidateIndex < candidates.length; candidateIndex++) {
|
||||
const candidate = candidates[candidateIndex];
|
||||
const isUsenet = ProwlarrService.isNZBResult(candidate);
|
||||
const protocol = isUsenet ? 'usenet' : 'torrent';
|
||||
const client = await manager.getClientServiceForProtocol(protocol);
|
||||
|
||||
logger.info(`Download added with ID: ${downloadClientId}`);
|
||||
if (!client) {
|
||||
throw new Error(`No ${protocol} download client configured. Please add a ${protocol} client in Settings > Download Clients.`);
|
||||
}
|
||||
|
||||
// Create DownloadHistory record
|
||||
// Determine indexer page URL - exclude magnet links from guid fallback
|
||||
const indexerPageUrl = torrent.infoUrl || (torrent.guid?.startsWith('magnet:') ? null : torrent.guid);
|
||||
const clientConfig = await manager.getClientForProtocol(protocol);
|
||||
const category = clientConfig?.category || 'readmeabook';
|
||||
|
||||
const downloadHistory = await prisma.downloadHistory.create({
|
||||
data: {
|
||||
logger.info(
|
||||
`Trying release ${candidateIndex + 1}/${candidates.length}: ${candidate.title}`,
|
||||
{ indexer: candidate.indexer, protocol, downloadClient: client.clientType }
|
||||
);
|
||||
|
||||
let downloadClientId: string;
|
||||
try {
|
||||
downloadClientId = await client.addDownload(candidate.downloadUrl, {
|
||||
category,
|
||||
priority: 'normal',
|
||||
sourceHeaders,
|
||||
});
|
||||
} catch (error) {
|
||||
if (!(error instanceof DownloadSourceError)) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
const contextualError = new DownloadSourceError(
|
||||
`Grab failed: ${candidate.indexer} returned HTTP ${error.status} while fetching "${candidate.title}" through Prowlarr`,
|
||||
error.status,
|
||||
error.sourceUrl,
|
||||
error
|
||||
);
|
||||
|
||||
if (!isRetryableDownloadSourceError(error)) {
|
||||
throw contextualError;
|
||||
}
|
||||
|
||||
const hasNextCandidate = candidateIndex < candidates.length - 1;
|
||||
|
||||
logger.warn(
|
||||
`${contextualError.message}${hasNextCandidate ? '; trying next ranked release' : ''}`,
|
||||
{
|
||||
indexer: candidate.indexer,
|
||||
upstreamStatus: error.status,
|
||||
candidate: candidateIndex + 1,
|
||||
candidateCount: candidates.length,
|
||||
}
|
||||
);
|
||||
|
||||
if (hasNextCandidate) {
|
||||
continue;
|
||||
}
|
||||
|
||||
throw contextualError;
|
||||
}
|
||||
|
||||
logger.info(`Download added with ID: ${downloadClientId}`);
|
||||
|
||||
// Create DownloadHistory record. Exclude magnet links from the indexer-page fallback.
|
||||
const indexerPageUrl = candidate.infoUrl || (candidate.guid?.startsWith('magnet:') ? null : candidate.guid);
|
||||
|
||||
const downloadHistory = await prisma.downloadHistory.create({
|
||||
data: {
|
||||
requestId,
|
||||
indexerName: candidate.indexer,
|
||||
indexerId: candidate.indexerId,
|
||||
downloadClient: client.clientType,
|
||||
downloadClientId,
|
||||
torrentName: candidate.title,
|
||||
// Set protocol-specific ID fields for backward compatibility
|
||||
torrentHash: client.protocol === 'torrent' ? (candidate.infoHash || downloadClientId) : undefined,
|
||||
nzbId: client.protocol === 'usenet' ? downloadClientId : undefined,
|
||||
torrentSizeBytes: candidate.size,
|
||||
torrentUrl: indexerPageUrl,
|
||||
magnetLink: candidate.downloadUrl,
|
||||
seeders: candidate.seeders || 0,
|
||||
leechers: candidate.leechers || 0,
|
||||
downloadStatus: 'downloading',
|
||||
selected: true,
|
||||
startedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
logger.info(`Created download history record: ${downloadHistory.id}`);
|
||||
|
||||
// Send grab notification (non-blocking — failures here don't fail the download)
|
||||
const jobQueue = getJobQueueService();
|
||||
const grabMessage = `${candidate.title} via ${candidate.indexer} (${client.clientType})`;
|
||||
await jobQueue.addNotificationJob(
|
||||
'request_grabbed',
|
||||
requestId,
|
||||
indexerName: torrent.indexer,
|
||||
indexerId: torrent.indexerId,
|
||||
downloadClient: client.clientType,
|
||||
audiobook.title,
|
||||
audiobook.author,
|
||||
request.user.plexUsername || 'Unknown User',
|
||||
grabMessage,
|
||||
request.type
|
||||
).catch((error) => {
|
||||
logger.error('Failed to queue grab notification', { error: error instanceof Error ? error.message : String(error) });
|
||||
});
|
||||
|
||||
// Trigger monitor download job with initial delay
|
||||
await jobQueue.addMonitorJob(
|
||||
requestId,
|
||||
downloadHistory.id,
|
||||
downloadClientId,
|
||||
torrentName: torrent.title,
|
||||
// Set protocol-specific ID fields for backward compatibility
|
||||
torrentHash: client.protocol === 'torrent' ? (torrent.infoHash || downloadClientId) : undefined,
|
||||
nzbId: client.protocol === 'usenet' ? downloadClientId : undefined,
|
||||
torrentSizeBytes: torrent.size,
|
||||
torrentUrl: indexerPageUrl,
|
||||
magnetLink: torrent.downloadUrl,
|
||||
seeders: torrent.seeders || 0,
|
||||
leechers: torrent.leechers || 0,
|
||||
downloadStatus: 'downloading',
|
||||
selected: true,
|
||||
startedAt: new Date(),
|
||||
},
|
||||
});
|
||||
client.clientType,
|
||||
3 // Wait 3 seconds before first check
|
||||
);
|
||||
|
||||
logger.info(`Created download history record: ${downloadHistory.id}`);
|
||||
logger.info(`Started monitoring job for request ${requestId} (${client.clientType}, 3s initial delay)`);
|
||||
|
||||
// Send grab notification (non-blocking — failures here don't fail the download)
|
||||
const jobQueue = getJobQueueService();
|
||||
const grabMessage = `${torrent.title} via ${torrent.indexer} (${client.clientType})`;
|
||||
await jobQueue.addNotificationJob(
|
||||
'request_grabbed',
|
||||
requestId,
|
||||
audiobook.title,
|
||||
audiobook.author,
|
||||
request.user.plexUsername || 'Unknown User',
|
||||
grabMessage,
|
||||
request.type
|
||||
).catch((error) => {
|
||||
logger.error('Failed to queue grab notification', { error: error instanceof Error ? error.message : String(error) });
|
||||
});
|
||||
return {
|
||||
success: true,
|
||||
message: `Download added to ${client.clientType} and monitoring started`,
|
||||
requestId,
|
||||
downloadHistoryId: downloadHistory.id,
|
||||
downloadClientId,
|
||||
torrent: {
|
||||
title: candidate.title,
|
||||
size: candidate.size,
|
||||
seeders: candidate.seeders || 0,
|
||||
format: candidate.format,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Trigger monitor download job with initial delay
|
||||
await jobQueue.addMonitorJob(
|
||||
requestId,
|
||||
downloadHistory.id,
|
||||
downloadClientId,
|
||||
client.clientType,
|
||||
3 // Wait 3 seconds before first check
|
||||
);
|
||||
|
||||
logger.info(`Started monitoring job for request ${requestId} (${client.clientType}, 3s initial delay)`);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
message: `Download added to ${client.clientType} and monitoring started`,
|
||||
requestId,
|
||||
downloadHistoryId: downloadHistory.id,
|
||||
downloadClientId,
|
||||
torrent: {
|
||||
title: torrent.title,
|
||||
size: torrent.size,
|
||||
seeders: torrent.seeders || 0,
|
||||
format: torrent.format,
|
||||
},
|
||||
};
|
||||
throw new Error('No release candidates were available to download');
|
||||
} catch (error) {
|
||||
logger.error(`Error: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
|
||||
if (isTransientConnectionError(error)) {
|
||||
if (isTransientConnectionError(error) || isRetryableDownloadSourceError(error)) {
|
||||
// Connection error — don't mark request as failed yet.
|
||||
// Bull will retry this job (3 attempts with exponential backoff).
|
||||
// If all retries are exhausted, the global failed handler marks it failed.
|
||||
logger.warn(`Download client unreachable for request ${requestId}, allowing Bull to retry`);
|
||||
logger.warn(
|
||||
isRetryableDownloadSourceError(error)
|
||||
? `All ranked release grabs were temporarily unavailable for request ${requestId}, allowing Bull to retry`
|
||||
: `Download client unreachable for request ${requestId}, allowing Bull to retry`
|
||||
);
|
||||
} else {
|
||||
// Permanent error — mark request as failed immediately
|
||||
await prisma.request.update({
|
||||
|
||||
@@ -17,6 +17,8 @@ export interface MonitorRssFeedsPayload {
|
||||
scheduledJobId?: string;
|
||||
}
|
||||
|
||||
const REQUEST_PAGE_SIZE = 100;
|
||||
|
||||
export async function processMonitorRssFeeds(payload: MonitorRssFeedsPayload): Promise<any> {
|
||||
const { jobId, scheduledJobId } = payload;
|
||||
const logger = RMABLogger.forJob(jobId, 'MonitorRssFeeds');
|
||||
@@ -63,16 +65,35 @@ export async function processMonitorRssFeeds(payload: MonitorRssFeedsPayload): P
|
||||
return { success: true, message: 'No RSS results', matched: 0 };
|
||||
}
|
||||
|
||||
// Get all active requests awaiting search (audiobooks and ebooks)
|
||||
// Both types can be matched against RSS torrent feeds
|
||||
const missingRequests = await prisma.request.findMany({
|
||||
// Get every active request awaiting search (audiobooks and ebooks) in
|
||||
// deterministic pages. A single unordered take(100) repeatedly inspected
|
||||
// the same rows and made requests beyond that batch invisible to RSS.
|
||||
const firstPage = await prisma.request.findMany({
|
||||
where: {
|
||||
status: 'awaiting_search',
|
||||
deletedAt: null,
|
||||
},
|
||||
include: { audiobook: true },
|
||||
take: 100,
|
||||
orderBy: { id: 'asc' },
|
||||
take: REQUEST_PAGE_SIZE,
|
||||
});
|
||||
const missingRequests = [...firstPage];
|
||||
let page = firstPage;
|
||||
|
||||
while (page.length === REQUEST_PAGE_SIZE) {
|
||||
page = await prisma.request.findMany({
|
||||
where: {
|
||||
status: 'awaiting_search',
|
||||
deletedAt: null,
|
||||
},
|
||||
include: { audiobook: true },
|
||||
orderBy: { id: 'asc' },
|
||||
cursor: { id: page[page.length - 1].id },
|
||||
skip: 1,
|
||||
take: REQUEST_PAGE_SIZE,
|
||||
});
|
||||
missingRequests.push(...page);
|
||||
}
|
||||
|
||||
logger.info(`Found ${missingRequests.length} requests awaiting search`);
|
||||
|
||||
|
||||
@@ -61,6 +61,11 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
|
||||
take: 1,
|
||||
},
|
||||
},
|
||||
orderBy: [
|
||||
{ lastImportAt: { sort: 'asc', nulls: 'first' } },
|
||||
{ createdAt: 'asc' },
|
||||
{ id: 'asc' },
|
||||
],
|
||||
take: 50, // Limit to 50 requests per run
|
||||
});
|
||||
|
||||
@@ -81,6 +86,13 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
|
||||
|
||||
for (const request of requests) {
|
||||
try {
|
||||
// Claim this retry slot immediately so every selected row rotates to
|
||||
// the back of the next batch, including malformed rows we must skip.
|
||||
await prisma.request.update({
|
||||
where: { id: request.id },
|
||||
data: { lastImportAt: new Date() },
|
||||
});
|
||||
|
||||
// Get the download path from the most recent download history
|
||||
const downloadHistory = request.downloadHistory[0];
|
||||
|
||||
|
||||
@@ -30,15 +30,36 @@ export async function processRetryMissingTorrents(payload: RetryMissingTorrentsP
|
||||
const configService = getConfigService();
|
||||
const skipUnreleasedSetting = (await configService.get('indexer.skip_unreleased')) !== 'false';
|
||||
|
||||
// Find all active requests in awaiting_search OR awaiting_release status
|
||||
// Release dates are stored as date-only values. When the release gate is
|
||||
// enabled, do not let still-future awaiting_release rows consume one of
|
||||
// the limited retry slots; they become eligible automatically on release day.
|
||||
const todayUtc = new Date();
|
||||
todayUtc.setUTCHours(0, 0, 0, 0);
|
||||
|
||||
// Process never-searched requests first, then rotate through the least
|
||||
// recently searched. Without an explicit order, PostgreSQL repeatedly
|
||||
// returned the same physical first 50 rows and starved the rest forever.
|
||||
const requests = await prisma.request.findMany({
|
||||
where: {
|
||||
status: { in: ['awaiting_search', 'awaiting_release'] },
|
||||
deletedAt: null,
|
||||
...(skipUnreleasedSetting
|
||||
? {
|
||||
OR: [
|
||||
{ status: 'awaiting_search' },
|
||||
{ status: 'awaiting_release', releaseDate: null },
|
||||
{ status: 'awaiting_release', releaseDate: { lte: todayUtc } },
|
||||
],
|
||||
}
|
||||
: { status: { in: ['awaiting_search', 'awaiting_release'] } }),
|
||||
},
|
||||
include: {
|
||||
audiobook: true,
|
||||
},
|
||||
orderBy: [
|
||||
{ lastSearchAt: { sort: 'asc', nulls: 'first' } },
|
||||
{ createdAt: 'asc' },
|
||||
{ id: 'asc' },
|
||||
],
|
||||
take: 50,
|
||||
});
|
||||
|
||||
|
||||
@@ -13,6 +13,8 @@ import { getLanguageForRegion } from '../constants/language-config';
|
||||
import { filterBlockedResults } from '../utils/filter-blocked-results';
|
||||
import type { AudibleRegion } from '../types/audible';
|
||||
|
||||
const MAX_RANKED_RESULTS = 100;
|
||||
|
||||
/**
|
||||
* Process search indexers job
|
||||
* Searches configured indexers for audiobook torrents
|
||||
@@ -103,7 +105,6 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
|
||||
categories: group.categories,
|
||||
indexerIds: group.indexerIds,
|
||||
minSeeders: 1, // Only torrents with at least 1 seeder
|
||||
maxResults: 100, // Limit per group
|
||||
});
|
||||
|
||||
logger.info(`Group ${i + 1} returned ${groupResults.length} results`);
|
||||
@@ -201,15 +202,20 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
|
||||
// Dual threshold filtering:
|
||||
// 1. Base score must be >= 50 (quality minimum)
|
||||
// 2. Final score must be >= 50 (not disqualified by negative bonuses)
|
||||
const filteredResults = rankedResults.filter(result =>
|
||||
const qualifyingResults = rankedResults.filter(result =>
|
||||
result.score >= 50 && result.finalScore >= 50
|
||||
);
|
||||
const resultsTruncated = qualifyingResults.length > MAX_RANKED_RESULTS;
|
||||
const filteredResults = qualifyingResults.slice(0, MAX_RANKED_RESULTS);
|
||||
|
||||
const disqualifiedByNegativeBonus = rankedResults.filter(result =>
|
||||
result.score >= 50 && result.finalScore < 50
|
||||
).length;
|
||||
|
||||
logger.info(`Ranked ${rankedResults.length} results, ${filteredResults.length} above threshold (50/100 base + final)`);
|
||||
logger.info(`Ranked ${rankedResults.length} results, ${qualifyingResults.length} above threshold (50/100 base + final)`);
|
||||
if (resultsTruncated) {
|
||||
logger.info(`Limited automatic search to the top ${MAX_RANKED_RESULTS} of ${qualifyingResults.length} qualifying results after ranking`);
|
||||
}
|
||||
if (disqualifiedByNegativeBonus > 0) {
|
||||
logger.info(`${disqualifiedByNegativeBonus} torrents disqualified by negative flag bonuses`);
|
||||
}
|
||||
@@ -283,7 +289,7 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
|
||||
id: audiobook.id,
|
||||
title: audiobook.title,
|
||||
author: audiobook.author,
|
||||
}, bestResult);
|
||||
}, bestResult, filteredResults.slice(1));
|
||||
|
||||
return {
|
||||
success: true,
|
||||
|
||||
@@ -58,6 +58,8 @@ export interface DownloadTorrentPayload extends JobPayload {
|
||||
author: string;
|
||||
};
|
||||
torrent: TorrentResult;
|
||||
/** Ranked candidates after `torrent`, used when an upstream grab is rate-limited/unavailable. */
|
||||
alternateTorrents?: TorrentResult[];
|
||||
}
|
||||
|
||||
export interface MonitorDownloadPayload extends JobPayload {
|
||||
@@ -599,7 +601,8 @@ export class JobQueueService {
|
||||
async addDownloadJob(
|
||||
requestId: string,
|
||||
audiobook: { id: string; title: string; author: string },
|
||||
torrent: TorrentResult
|
||||
torrent: TorrentResult,
|
||||
alternateTorrents: TorrentResult[] = []
|
||||
): Promise<string> {
|
||||
return await this.addJob(
|
||||
'download_torrent',
|
||||
@@ -607,6 +610,7 @@ export class JobQueueService {
|
||||
requestId,
|
||||
audiobook,
|
||||
torrent,
|
||||
alternateTorrents,
|
||||
} as DownloadTorrentPayload,
|
||||
{
|
||||
priority: 9, // High priority - download selected torrent
|
||||
|
||||
@@ -9,7 +9,6 @@ import { prisma } from '../db';
|
||||
import * as fs from 'fs/promises';
|
||||
import * as path from 'path';
|
||||
import { RMABLogger } from '../utils/logger';
|
||||
import { buildAudiobookPath } from '../utils/file-organizer';
|
||||
import { CLIENT_PROTOCOL_MAP, DownloadClientType } from '../interfaces/download-client.interface';
|
||||
|
||||
const logger = RMABLogger.create('RequestDelete');
|
||||
@@ -66,6 +65,7 @@ export async function deleteRequest(
|
||||
audibleAsin: true,
|
||||
plexGuid: true,
|
||||
absItemId: true,
|
||||
filePath: true,
|
||||
fileFormat: true,
|
||||
},
|
||||
},
|
||||
@@ -202,72 +202,46 @@ export async function deleteRequest(
|
||||
// For ebooks: delete only ebook files (leave audiobook files intact)
|
||||
let filesDeleted = false;
|
||||
try {
|
||||
const { getConfigService } = await import('./config.service');
|
||||
const configService = getConfigService();
|
||||
const mediaDir = (await configService.get('media_dir')) || '/media/audiobooks';
|
||||
// Use ebook-specific template for ebook requests, with fallback to audiobook template
|
||||
const audiobookTemplate = (await configService.get('audiobook_path_template')) || '{author}/{title} {asin}';
|
||||
const template = isEbook
|
||||
? (await configService.get('ebook_path_template')) || audiobookTemplate
|
||||
: audiobookTemplate;
|
||||
const titleFolderPath = request.audiobook.filePath;
|
||||
|
||||
// Fetch year from audible cache if ASIN is available
|
||||
let year: number | undefined;
|
||||
if (request.audiobook.audibleAsin) {
|
||||
const audibleCache = await prisma.audibleCache.findUnique({
|
||||
where: { asin: request.audiobook.audibleAsin },
|
||||
select: { releaseDate: true },
|
||||
});
|
||||
if (audibleCache?.releaseDate) {
|
||||
year = new Date(audibleCache.releaseDate).getFullYear();
|
||||
}
|
||||
}
|
||||
if (!titleFolderPath) {
|
||||
logger.warn(
|
||||
`Skipping media file deletion for request ${requestId}: no stored file path`
|
||||
);
|
||||
} else {
|
||||
// Check if the directory recorded during organization still exists.
|
||||
try {
|
||||
await fs.access(titleFolderPath);
|
||||
|
||||
// Build path using centralized function
|
||||
const titleFolderPath = buildAudiobookPath(
|
||||
mediaDir,
|
||||
template,
|
||||
{
|
||||
author: request.audiobook.author,
|
||||
title: request.audiobook.title,
|
||||
narrator: request.audiobook.narrator || undefined,
|
||||
asin: request.audiobook.audibleAsin || undefined,
|
||||
year,
|
||||
}
|
||||
);
|
||||
if (isEbook) {
|
||||
// For ebooks: only delete ebook files, leave audiobook files intact
|
||||
const ebookExtensions = ['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.fb2', '.cbz', '.cbr'];
|
||||
const files = await fs.readdir(titleFolderPath);
|
||||
|
||||
// Check if folder exists
|
||||
try {
|
||||
await fs.access(titleFolderPath);
|
||||
|
||||
if (isEbook) {
|
||||
// For ebooks: only delete ebook files, leave audiobook files intact
|
||||
const ebookExtensions = ['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.fb2', '.cbz', '.cbr'];
|
||||
const files = await fs.readdir(titleFolderPath);
|
||||
|
||||
let deletedCount = 0;
|
||||
for (const file of files) {
|
||||
const ext = path.extname(file).toLowerCase();
|
||||
if (ebookExtensions.includes(ext)) {
|
||||
const filePath = path.join(titleFolderPath, file);
|
||||
await fs.unlink(filePath);
|
||||
logger.info(`Deleted ebook file: ${file}`);
|
||||
deletedCount++;
|
||||
let deletedCount = 0;
|
||||
for (const file of files) {
|
||||
const ext = path.extname(file).toLowerCase();
|
||||
if (ebookExtensions.includes(ext)) {
|
||||
const filePath = path.join(titleFolderPath, file);
|
||||
await fs.unlink(filePath);
|
||||
logger.info(`Deleted ebook file: ${file}`);
|
||||
deletedCount++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
filesDeleted = deletedCount > 0;
|
||||
logger.info(`Deleted ${deletedCount} ebook file(s) from: ${titleFolderPath}`);
|
||||
} else {
|
||||
// For audiobooks: delete the entire title folder
|
||||
await fs.rm(titleFolderPath, { recursive: true, force: true });
|
||||
logger.info(`Deleted media directory: ${titleFolderPath}`);
|
||||
filesDeleted = true;
|
||||
filesDeleted = deletedCount > 0;
|
||||
logger.info(`Deleted ${deletedCount} ebook file(s) from: ${titleFolderPath}`);
|
||||
} else {
|
||||
// For audiobooks: delete the entire title folder
|
||||
await fs.rm(titleFolderPath, { recursive: true, force: true });
|
||||
logger.info(`Deleted media directory: ${titleFolderPath}`);
|
||||
filesDeleted = true;
|
||||
}
|
||||
} catch (accessError) {
|
||||
// Folder doesn't exist - that's okay
|
||||
logger.info(`Media directory not found: ${titleFolderPath}`);
|
||||
filesDeleted = false;
|
||||
}
|
||||
} catch (accessError) {
|
||||
// Folder doesn't exist - that's okay
|
||||
logger.info(`Media directory not found: ${titleFolderPath}`);
|
||||
filesDeleted = false;
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
|
||||
@@ -20,6 +20,10 @@ const rankTorrentsMock = vi.hoisted(() => vi.fn());
|
||||
const groupIndexersMock = vi.hoisted(() => vi.fn());
|
||||
const groupDescriptionMock = vi.hoisted(() => vi.fn(() => 'Group'));
|
||||
|
||||
vi.mock('@/lib/db', () => ({
|
||||
prisma: {},
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/middleware/auth', () => ({
|
||||
requireAuth: requireAuthMock,
|
||||
}));
|
||||
@@ -94,6 +98,45 @@ describe('Audiobooks search torrents route', () => {
|
||||
expect(payload.results[0].rank).toBe(1);
|
||||
expect(rankTorrentsMock).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('ranks all candidates before returning the top 100 with truncation metadata', async () => {
|
||||
authRequest.json.mockResolvedValue({ title: 'Title', author: 'Author' });
|
||||
configServiceMock.get
|
||||
.mockResolvedValueOnce(JSON.stringify([{ id: 1, name: 'Indexer', protocol: 'torrent', priority: 10 }]))
|
||||
.mockResolvedValueOnce(null);
|
||||
groupIndexersMock.mockReturnValue({ groups: [{ categories: [3030], indexerIds: [1] }], skippedIndexers: [] });
|
||||
|
||||
const candidates = Array.from({ length: 101 }, (_, index) => ({
|
||||
title: index === 100 ? 'Best result' : `Result ${index}`,
|
||||
size: 100,
|
||||
indexer: 'Indexer',
|
||||
indexerId: 1,
|
||||
}));
|
||||
const rankedCandidates = [candidates[100], ...candidates.slice(0, 100)].map((result, index) => ({
|
||||
...result,
|
||||
score: 100 - index,
|
||||
breakdown: { matchScore: 50, formatScore: 0, sizeScore: 0, seederScore: 0, notes: [] },
|
||||
bonusPoints: 0,
|
||||
bonusModifiers: [],
|
||||
finalScore: 100 - index,
|
||||
}));
|
||||
prowlarrMock.searchWithVariations.mockResolvedValue(candidates);
|
||||
rankTorrentsMock.mockReturnValue(rankedCandidates);
|
||||
|
||||
const { POST } = await import('@/app/api/audiobooks/search-torrents/route');
|
||||
const response = await POST({} as any);
|
||||
const payload = await response.json();
|
||||
|
||||
expect(rankTorrentsMock.mock.calls[0][0]).toHaveLength(101);
|
||||
expect(prowlarrMock.searchWithVariations).toHaveBeenCalledWith('Title', 'Author', {
|
||||
categories: [3030],
|
||||
indexerIds: [1],
|
||||
});
|
||||
expect(payload.results).toHaveLength(100);
|
||||
expect(payload.results[0]).toEqual(expect.objectContaining({ title: 'Best result', rank: 1 }));
|
||||
expect(payload.truncated).toBe(true);
|
||||
expect(payload.rawCount).toBe(101);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
|
||||
@@ -153,6 +153,51 @@ describe('Request action routes', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('ranks the complete interactive result set before returning the top 100', async () => {
|
||||
authRequest.json.mockResolvedValue({});
|
||||
prismaMock.request.findUnique.mockResolvedValueOnce({
|
||||
id: 'req-complete-ranking',
|
||||
userId: 'user-1',
|
||||
audiobook: { title: 'Title', author: 'Author', audibleAsin: null },
|
||||
});
|
||||
prismaMock.user.findUnique.mockResolvedValueOnce({
|
||||
role: 'user',
|
||||
interactiveSearchAccess: null,
|
||||
});
|
||||
configServiceMock.get.mockResolvedValueOnce(JSON.stringify([{ id: 1, priority: 10, categories: [3030] }]));
|
||||
configServiceMock.get.mockResolvedValueOnce(null);
|
||||
groupIndexersMock.mockReturnValue({ groups: [{ categories: [3030], indexerIds: [1] }], skippedIndexers: [] });
|
||||
|
||||
const candidates = Array.from({ length: 101 }, (_, index) => ({
|
||||
title: index === 100 ? 'Best result' : `Result ${index}`,
|
||||
size: 100,
|
||||
}));
|
||||
const rankedCandidates = [candidates[100], ...candidates.slice(0, 100)].map((result, index) => ({
|
||||
...result,
|
||||
score: 100 - index,
|
||||
breakdown: { matchScore: 50, formatScore: 0, sizeScore: 0, seederScore: 0, notes: [] },
|
||||
bonusPoints: 0,
|
||||
bonusModifiers: [],
|
||||
finalScore: 100 - index,
|
||||
}));
|
||||
prowlarrMock.searchWithVariations.mockResolvedValueOnce(candidates);
|
||||
rankTorrentsMock.mockReturnValueOnce(rankedCandidates);
|
||||
|
||||
const { POST } = await import('@/app/api/requests/[id]/interactive-search/route');
|
||||
const response = await POST({} as any, { params: Promise.resolve({ id: 'req-complete-ranking' }) });
|
||||
const payload = await response.json();
|
||||
|
||||
expect(rankTorrentsMock.mock.calls[0][0]).toHaveLength(101);
|
||||
expect(prowlarrMock.searchWithVariations).toHaveBeenCalledWith('Title', 'Author', {
|
||||
categories: [3030],
|
||||
indexerIds: [1],
|
||||
});
|
||||
expect(payload.results).toHaveLength(100);
|
||||
expect(payload.results[0]).toEqual(expect.objectContaining({ title: 'Best result', rank: 1 }));
|
||||
expect(payload.truncated).toBe(true);
|
||||
expect(payload.rawCount).toBe(101);
|
||||
});
|
||||
|
||||
it('performs interactive search gracefully when runtime fetch fails', async () => {
|
||||
authRequest.json.mockResolvedValue({});
|
||||
prismaMock.request.findUnique.mockResolvedValueOnce({
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import path from 'path';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { QBittorrentService, getQBittorrentService, invalidateQBittorrentService } from '@/lib/integrations/qbittorrent.service';
|
||||
import { DownloadSourceError } from '@/lib/interfaces/download-client.interface';
|
||||
|
||||
const clientMock = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
@@ -693,6 +694,32 @@ describe('QBittorrentService', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('preserves upstream HTTP status and forwards source headers', async () => {
|
||||
const service = new QBittorrentService('http://qb', 'user', 'pass');
|
||||
(service as any).cookie = 'SID=source-error';
|
||||
vi.spyOn(service as any, 'ensureCategory').mockResolvedValue(undefined);
|
||||
|
||||
axiosMock.get.mockRejectedValueOnce({
|
||||
isAxiosError: true,
|
||||
response: { status: 500, headers: {} },
|
||||
message: 'Request failed',
|
||||
});
|
||||
|
||||
const error = await service.addDownload(
|
||||
'https://prowlarr/1/download/limited',
|
||||
{ sourceHeaders: { 'X-Api-Key': 'secret' } }
|
||||
).catch((caught) => caught);
|
||||
|
||||
expect(error).toBeInstanceOf(DownloadSourceError);
|
||||
expect(error).toMatchObject({ status: 500 });
|
||||
expect(axiosMock.get).toHaveBeenCalledWith(
|
||||
'https://prowlarr/1/download/limited',
|
||||
expect.objectContaining({
|
||||
headers: expect.objectContaining({ 'X-Api-Key': 'secret' }),
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
it('throws for invalid redirect locations when fetching torrents', async () => {
|
||||
const service = new QBittorrentService('http://qb', 'user', 'pass');
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { createPrismaMock } from '../helpers/prisma';
|
||||
import { createJobQueueMock } from '../helpers/job-queue';
|
||||
import { DownloadSourceError } from '@/lib/interfaces/download-client.interface';
|
||||
|
||||
const prismaMock = createPrismaMock();
|
||||
const configMock = vi.hoisted(() => ({
|
||||
@@ -214,4 +215,107 @@ describe('processDownloadTorrent', () => {
|
||||
|
||||
expect(downloadClientManagerMock.getClientServiceForProtocol).toHaveBeenCalledWith('usenet');
|
||||
});
|
||||
|
||||
it('tries the next ranked release when an indexer grab returns 500', async () => {
|
||||
const fallbackTorrent = {
|
||||
...torrentPayload.torrent,
|
||||
indexer: 'Healthy Indexer',
|
||||
indexerId: 2,
|
||||
title: 'Book - Author - Fallback',
|
||||
downloadUrl: 'https://prowlarr/2/download/fallback',
|
||||
guid: 'guid-fallback',
|
||||
};
|
||||
const qbtClientMock = {
|
||||
clientType: 'qbittorrent',
|
||||
protocol: 'torrent',
|
||||
addDownload: vi.fn()
|
||||
.mockRejectedValueOnce(new DownloadSourceError(
|
||||
'Grab failed: source returned HTTP 500',
|
||||
500,
|
||||
'https://prowlarr/1/download/limited'
|
||||
))
|
||||
.mockResolvedValueOnce('fallback-hash'),
|
||||
};
|
||||
downloadClientManagerMock.getClientServiceForProtocol.mockResolvedValue(qbtClientMock);
|
||||
downloadClientManagerMock.getClientForProtocol.mockResolvedValue({
|
||||
id: 'client-1',
|
||||
type: 'qbittorrent',
|
||||
enabled: true,
|
||||
category: 'readmeabook',
|
||||
});
|
||||
prismaMock.request.update.mockResolvedValue({
|
||||
type: 'audiobook',
|
||||
user: { plexUsername: 'testuser' },
|
||||
});
|
||||
prismaMock.downloadHistory.create.mockResolvedValue({ id: 'dh-fallback' });
|
||||
|
||||
const { processDownloadTorrent } = await import('@/lib/processors/download-torrent.processor');
|
||||
const result = await processDownloadTorrent({
|
||||
...torrentPayload,
|
||||
torrent: {
|
||||
...torrentPayload.torrent,
|
||||
downloadUrl: 'https://prowlarr/1/download/limited',
|
||||
},
|
||||
alternateTorrents: [fallbackTorrent],
|
||||
});
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.torrent.title).toBe(fallbackTorrent.title);
|
||||
expect(qbtClientMock.addDownload).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
'https://prowlarr/1/download/limited',
|
||||
expect.anything()
|
||||
);
|
||||
expect(qbtClientMock.addDownload).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
fallbackTorrent.downloadUrl,
|
||||
expect.anything()
|
||||
);
|
||||
expect(prismaMock.downloadHistory.create).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
data: expect.objectContaining({
|
||||
indexerName: 'Healthy Indexer',
|
||||
torrentName: fallbackTorrent.title,
|
||||
downloadClientId: 'fallback-hash',
|
||||
}),
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves the request retryable when every ranked grab is temporarily unavailable', async () => {
|
||||
const qbtClientMock = {
|
||||
clientType: 'qbittorrent',
|
||||
protocol: 'torrent',
|
||||
addDownload: vi.fn().mockRejectedValue(new DownloadSourceError(
|
||||
'Grab failed: source returned HTTP 429',
|
||||
429,
|
||||
'https://prowlarr/1/download/limited'
|
||||
)),
|
||||
};
|
||||
downloadClientManagerMock.getClientServiceForProtocol.mockResolvedValue(qbtClientMock);
|
||||
downloadClientManagerMock.getClientForProtocol.mockResolvedValue({
|
||||
id: 'client-1',
|
||||
type: 'qbittorrent',
|
||||
enabled: true,
|
||||
category: 'readmeabook',
|
||||
});
|
||||
prismaMock.request.update.mockResolvedValue({
|
||||
type: 'audiobook',
|
||||
user: { plexUsername: 'testuser' },
|
||||
});
|
||||
|
||||
const { processDownloadTorrent } = await import('@/lib/processors/download-torrent.processor');
|
||||
await expect(processDownloadTorrent({
|
||||
...torrentPayload,
|
||||
torrent: {
|
||||
...torrentPayload.torrent,
|
||||
downloadUrl: 'https://prowlarr/1/download/limited',
|
||||
},
|
||||
})).rejects.toThrow('Indexer returned HTTP 429');
|
||||
|
||||
expect(prismaMock.request.update).not.toHaveBeenCalledWith(
|
||||
expect.objectContaining({ data: expect.objectContaining({ status: 'failed' }) })
|
||||
);
|
||||
expect(prismaMock.downloadHistory.create).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -34,6 +34,7 @@ function futureDate(days = 30): Date {
|
||||
|
||||
describe('processMonitorRssFeeds', () => {
|
||||
beforeEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.clearAllMocks();
|
||||
// Default to empty blocklist so the filter is a no-op unless a test overrides.
|
||||
prismaMock.blockedRelease.findMany.mockResolvedValue([]);
|
||||
@@ -73,6 +74,70 @@ describe('processMonitorRssFeeds', () => {
|
||||
'req-1',
|
||||
expect.objectContaining({ title: 'Great Book', author: 'Author Name' })
|
||||
);
|
||||
expect(prismaMock.request.findMany).toHaveBeenCalledWith(expect.objectContaining({
|
||||
orderBy: { id: 'asc' },
|
||||
take: 100,
|
||||
}));
|
||||
});
|
||||
|
||||
it('pages beyond the first 100 awaiting requests', async () => {
|
||||
vi.useFakeTimers();
|
||||
configMock.get.mockImplementation(async (key: string) => {
|
||||
if (key === 'prowlarr_indexers') {
|
||||
return JSON.stringify([{ id: 1, name: 'Indexer', rssEnabled: true }]);
|
||||
}
|
||||
return null;
|
||||
});
|
||||
prowlarrMock.getAllRssFeeds.mockResolvedValue([
|
||||
{ title: 'Target Story - Target Author' },
|
||||
]);
|
||||
|
||||
const firstPage = Array.from({ length: 100 }, (_, index) => ({
|
||||
id: `req-${String(index).padStart(3, '0')}`,
|
||||
type: 'audiobook',
|
||||
status: 'awaiting_search',
|
||||
releaseDate: null,
|
||||
audiobook: {
|
||||
id: `a-${index}`,
|
||||
title: `Unrelated Volume ${index}`,
|
||||
author: 'Someone Else',
|
||||
audibleAsin: `ASIN-${index}`,
|
||||
},
|
||||
}));
|
||||
const secondPage = [{
|
||||
id: 'req-target',
|
||||
type: 'audiobook',
|
||||
status: 'awaiting_search',
|
||||
releaseDate: null,
|
||||
audiobook: {
|
||||
id: 'a-target',
|
||||
title: 'Target Story',
|
||||
author: 'Target Author',
|
||||
audibleAsin: 'ASIN-TARGET',
|
||||
},
|
||||
}];
|
||||
prismaMock.request.findMany
|
||||
.mockResolvedValueOnce(firstPage)
|
||||
.mockResolvedValueOnce(secondPage);
|
||||
|
||||
const { processMonitorRssFeeds } = await import('@/lib/processors/monitor-rss-feeds.processor');
|
||||
const processing = processMonitorRssFeeds({ jobId: 'job-paged' });
|
||||
await vi.runAllTimersAsync();
|
||||
const result = await processing;
|
||||
|
||||
expect(result.totalMissing).toBe(101);
|
||||
expect(result.matched).toBe(1);
|
||||
expect(prismaMock.request.findMany).toHaveBeenNthCalledWith(2, expect.objectContaining({
|
||||
orderBy: { id: 'asc' },
|
||||
cursor: { id: 'req-099' },
|
||||
skip: 1,
|
||||
take: 100,
|
||||
}));
|
||||
expect(jobQueueMock.addSearchJob).toHaveBeenCalledWith(
|
||||
'req-target',
|
||||
expect.objectContaining({ title: 'Target Story', author: 'Target Author' })
|
||||
);
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it('skips RSS auto-search when matched book is unreleased and setting ON', async () => {
|
||||
|
||||
@@ -77,6 +77,18 @@ describe('processRetryFailedImports', () => {
|
||||
const result = await processRetryFailedImports({ jobId: 'job-1' });
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(prismaMock.request.findMany).toHaveBeenCalledWith(expect.objectContaining({
|
||||
orderBy: [
|
||||
{ lastImportAt: { sort: 'asc', nulls: 'first' } },
|
||||
{ createdAt: 'asc' },
|
||||
{ id: 'asc' },
|
||||
],
|
||||
take: 50,
|
||||
}));
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'req-1' },
|
||||
data: { lastImportAt: expect.any(Date) },
|
||||
});
|
||||
expect(jobQueueMock.addOrganizeJob).toHaveBeenCalledWith(
|
||||
'req-1',
|
||||
'a1',
|
||||
@@ -109,6 +121,10 @@ describe('processRetryFailedImports', () => {
|
||||
|
||||
expect(result.skipped).toBe(1);
|
||||
expect(result.triggered).toBe(0);
|
||||
expect(prismaMock.request.update).toHaveBeenCalledWith({
|
||||
where: { id: 'req-2' },
|
||||
data: { lastImportAt: expect.any(Date) },
|
||||
});
|
||||
});
|
||||
|
||||
it('falls back to configured download dir when qBittorrent lookup fails', async () => {
|
||||
|
||||
@@ -53,6 +53,23 @@ describe('processRetryMissingTorrents', () => {
|
||||
const result = await processRetryMissingTorrents({ jobId: 'job-1' });
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(prismaMock.request.findMany).toHaveBeenCalledWith({
|
||||
where: {
|
||||
deletedAt: null,
|
||||
OR: [
|
||||
{ status: 'awaiting_search' },
|
||||
{ status: 'awaiting_release', releaseDate: null },
|
||||
{ status: 'awaiting_release', releaseDate: { lte: expect.any(Date) } },
|
||||
],
|
||||
},
|
||||
include: { audiobook: true },
|
||||
orderBy: [
|
||||
{ lastSearchAt: { sort: 'asc', nulls: 'first' } },
|
||||
{ createdAt: 'asc' },
|
||||
{ id: 'asc' },
|
||||
],
|
||||
take: 50,
|
||||
});
|
||||
expect(jobQueueMock.addSearchJob).toHaveBeenCalledWith(
|
||||
'req-1',
|
||||
expect.objectContaining({ id: 'a1', title: 'Book', author: 'Author' })
|
||||
@@ -155,5 +172,11 @@ describe('processRetryMissingTorrents', () => {
|
||||
expect(prismaMock.request.update).not.toHaveBeenCalled();
|
||||
expect(jobQueueMock.addSearchJob).toHaveBeenCalled();
|
||||
expect(result.triggered).toBe(1);
|
||||
expect(prismaMock.request.findMany).toHaveBeenCalledWith(expect.objectContaining({
|
||||
where: {
|
||||
deletedAt: null,
|
||||
status: { in: ['awaiting_search', 'awaiting_release'] },
|
||||
},
|
||||
}));
|
||||
});
|
||||
});
|
||||
|
||||
@@ -88,6 +88,17 @@ describe('processSearchIndexers', () => {
|
||||
guid: 'guid-1',
|
||||
format: 'M4B',
|
||||
},
|
||||
{
|
||||
indexer: 'Fallback Indexer',
|
||||
indexerId: 2,
|
||||
title: 'Book - Author - Alternate',
|
||||
size: 50 * 1024 * 1024,
|
||||
seeders: 1,
|
||||
publishDate: new Date(),
|
||||
downloadUrl: 'https://prowlarr/2/download/alternate',
|
||||
guid: 'guid-2',
|
||||
format: 'M4B',
|
||||
},
|
||||
]);
|
||||
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
@@ -103,8 +114,55 @@ describe('processSearchIndexers', () => {
|
||||
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
|
||||
'req-2',
|
||||
{ id: 'a2', title: 'Book', author: 'Author' },
|
||||
expect.objectContaining({ title: 'Book - Author' })
|
||||
expect.objectContaining({ title: 'Book - Author' }),
|
||||
[expect.objectContaining({ title: 'Book - Author - Alternate' })]
|
||||
);
|
||||
expect(prowlarrMock.searchWithVariations).toHaveBeenCalledWith('Book', 'Author', {
|
||||
categories: [3030],
|
||||
indexerIds: [1],
|
||||
minSeeders: 1,
|
||||
});
|
||||
});
|
||||
|
||||
it('ranks all candidates before limiting automatic search to 100 results', async () => {
|
||||
configMock.get.mockImplementation(async (key: string) => {
|
||||
if (key === 'prowlarr_indexers') {
|
||||
return JSON.stringify([{ id: 1, name: 'Indexer', protocol: 'torrent', priority: 10, categories: [3030] }]);
|
||||
}
|
||||
if (key === 'indexer_flag_config') return JSON.stringify([]);
|
||||
return null;
|
||||
});
|
||||
|
||||
const candidates = Array.from({ length: 101 }, (_, index) => ({
|
||||
indexer: 'Indexer',
|
||||
indexerId: 1,
|
||||
title: 'Book - Author',
|
||||
size: 50 * 1024 * 1024,
|
||||
seeders: index === 100 ? 500 : 1,
|
||||
publishDate: new Date(),
|
||||
downloadUrl: `magnet:?xt=urn:btih:${index}`,
|
||||
guid: `guid-${index}`,
|
||||
format: 'M4B',
|
||||
}));
|
||||
prowlarrMock.searchWithVariations.mockResolvedValue(candidates);
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
|
||||
const { processSearchIndexers } = await import('@/lib/processors/search-indexers.processor');
|
||||
const result = await processSearchIndexers({
|
||||
requestId: 'req-rank-before-limit',
|
||||
audiobook: { id: 'a-rank-before-limit', title: 'Book', author: 'Author' },
|
||||
jobId: 'job-rank-before-limit',
|
||||
});
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
|
||||
'req-rank-before-limit',
|
||||
expect.anything(),
|
||||
expect.objectContaining({ guid: 'guid-100' }),
|
||||
expect.any(Array)
|
||||
);
|
||||
const fallbackResults = jobQueueMock.addDownloadJob.mock.calls[0][3];
|
||||
expect(fallbackResults).toHaveLength(99);
|
||||
});
|
||||
|
||||
it('fails when no indexers are configured', async () => {
|
||||
@@ -179,7 +237,8 @@ describe('processSearchIndexers', () => {
|
||||
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
|
||||
'req-filter-name',
|
||||
expect.objectContaining({ id: 'a-filter' }),
|
||||
expect.objectContaining({ title: 'Good Release - Author' })
|
||||
expect.objectContaining({ title: 'Good Release - Author' }),
|
||||
[]
|
||||
);
|
||||
});
|
||||
|
||||
@@ -235,7 +294,8 @@ describe('processSearchIndexers', () => {
|
||||
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
|
||||
'req-filter-hash',
|
||||
expect.anything(),
|
||||
expect.objectContaining({ title: 'Good Release - Author' })
|
||||
expect.objectContaining({ title: 'Good Release - Author' }),
|
||||
[]
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -202,11 +202,20 @@ describe('JobQueueService', () => {
|
||||
|
||||
const { JobQueueService } = await import('@/lib/services/job-queue.service');
|
||||
const service = new JobQueueService();
|
||||
await service.addDownloadJob('req-1', { id: 'ab-1', title: 'Title', author: 'Author' }, { hash: 'hash' } as any);
|
||||
await service.addDownloadJob(
|
||||
'req-1',
|
||||
{ id: 'ab-1', title: 'Title', author: 'Author' },
|
||||
{ hash: 'hash' } as any,
|
||||
[{ hash: 'fallback-hash' } as any]
|
||||
);
|
||||
|
||||
expect(queueMock.add).toHaveBeenCalledWith(
|
||||
'download_torrent',
|
||||
expect.objectContaining({ requestId: 'req-1', jobId: 'job-2' }),
|
||||
expect.objectContaining({
|
||||
requestId: 'req-1',
|
||||
jobId: 'job-2',
|
||||
alternateTorrents: [{ hash: 'fallback-hash' }],
|
||||
}),
|
||||
expect.objectContaining({ priority: 9 })
|
||||
);
|
||||
});
|
||||
|
||||
@@ -38,13 +38,6 @@ vi.mock('@/lib/services/audiobookshelf/api', () => ({
|
||||
deleteABSItem: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/utils/file-organizer', () => ({
|
||||
buildAudiobookPath: vi.fn((mediaDir: string, template: string, data: any) => {
|
||||
// Simple mock implementation that mimics the real behavior for tests
|
||||
return path.join(mediaDir, data.author, `${data.title} ${data.asin}`);
|
||||
}),
|
||||
}));
|
||||
|
||||
describe('deleteRequest', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
@@ -74,6 +67,7 @@ describe('deleteRequest', () => {
|
||||
audibleAsin: 'ASIN1',
|
||||
plexGuid: 'plex-1',
|
||||
absItemId: null,
|
||||
filePath: path.join('/media', 'Author', 'Book ASIN1'),
|
||||
},
|
||||
downloadHistory: [
|
||||
{
|
||||
@@ -154,6 +148,7 @@ describe('deleteRequest', () => {
|
||||
audibleAsin: null,
|
||||
plexGuid: 'plex-2',
|
||||
absItemId: null,
|
||||
filePath: path.join('/media', 'Author', 'Book Two'),
|
||||
},
|
||||
downloadHistory: [
|
||||
{
|
||||
@@ -219,6 +214,7 @@ describe('deleteRequest', () => {
|
||||
audibleAsin: 'ASIN3',
|
||||
plexGuid: 'plex-3',
|
||||
absItemId: null,
|
||||
filePath: path.join('/media', 'Author Name', 'Book Three ASIN3'),
|
||||
},
|
||||
downloadHistory: [
|
||||
{
|
||||
@@ -291,6 +287,7 @@ describe('deleteRequest', () => {
|
||||
audibleAsin: null,
|
||||
plexGuid: 'plex-4',
|
||||
absItemId: null,
|
||||
filePath: path.join('/media', 'Author', 'Book Four'),
|
||||
},
|
||||
downloadHistory: [
|
||||
{
|
||||
@@ -349,6 +346,7 @@ describe('deleteRequest', () => {
|
||||
audibleAsin: null,
|
||||
plexGuid: null,
|
||||
absItemId: 'abs-5',
|
||||
filePath: path.join('/media', 'Author', 'Book Five'),
|
||||
},
|
||||
downloadHistory: [
|
||||
{
|
||||
@@ -401,6 +399,7 @@ describe('deleteRequest', () => {
|
||||
audibleAsin: 'ASIN6',
|
||||
plexGuid: null,
|
||||
absItemId: 'abs-item-123',
|
||||
filePath: path.join('/media', 'Author Six', 'Book Six ASIN6'),
|
||||
},
|
||||
downloadHistory: [],
|
||||
});
|
||||
@@ -447,6 +446,7 @@ describe('deleteRequest', () => {
|
||||
audibleAsin: null,
|
||||
plexGuid: null,
|
||||
absItemId: 'abs-item-456',
|
||||
filePath: path.join('/media', 'Author Seven', 'Book Seven'),
|
||||
},
|
||||
downloadHistory: [],
|
||||
});
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
/**
|
||||
* Component: Request Delete Stored Path Regression Tests
|
||||
* Documentation: documentation/admin-features/request-deletion.md
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import * as fs from 'fs/promises';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { createPrismaMock } from '../helpers/prisma';
|
||||
|
||||
const prismaMock = createPrismaMock();
|
||||
const configServiceMock = {
|
||||
get: vi.fn(),
|
||||
getBackendMode: vi.fn(),
|
||||
};
|
||||
|
||||
vi.mock('@/lib/db', () => ({
|
||||
prisma: prismaMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/services/config.service', () => ({
|
||||
getConfigService: () => configServiceMock,
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/utils/logger', () => ({
|
||||
RMABLogger: {
|
||||
create: () => ({
|
||||
info: vi.fn(),
|
||||
warn: vi.fn(),
|
||||
error: vi.fn(),
|
||||
}),
|
||||
},
|
||||
}));
|
||||
|
||||
describe('deleteRequest stored media path', () => {
|
||||
let tempRoot: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.clearAllMocks();
|
||||
tempRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'rmab-request-delete-'));
|
||||
configServiceMock.get.mockRejectedValue(
|
||||
new Error('Media deletion must not read current path configuration')
|
||||
);
|
||||
configServiceMock.getBackendMode.mockResolvedValue('plex');
|
||||
prismaMock.request.findMany.mockResolvedValue([]);
|
||||
prismaMock.request.updateMany.mockResolvedValue({ count: 0 });
|
||||
prismaMock.request.update.mockResolvedValue({});
|
||||
prismaMock.audiobook.update.mockResolvedValue({});
|
||||
prismaMock.plexLibrary.findMany.mockResolvedValue([]);
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fs.rm(tempRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('deletes the stored directory and preserves the path rendered by the current template', async () => {
|
||||
const storedPath = path.join(tempRoot, 'Arthur Conan Doyle', 'The Valley of Fear');
|
||||
const currentTemplatePath = path.join(
|
||||
tempRoot,
|
||||
'Arthur Conan Doyle',
|
||||
'1914 - The Valley of Fear'
|
||||
);
|
||||
const unrelatedFile = path.join(currentTemplatePath, 'my own rip.m4b');
|
||||
|
||||
await fs.mkdir(storedPath, { recursive: true });
|
||||
await fs.mkdir(currentTemplatePath, { recursive: true });
|
||||
await fs.writeFile(path.join(storedPath, 'book.m4b'), 'organized by RMAB');
|
||||
await fs.writeFile(unrelatedFile, 'user-owned file');
|
||||
|
||||
prismaMock.request.findFirst.mockResolvedValue({
|
||||
id: 'req-stored-path',
|
||||
type: 'audiobook',
|
||||
audiobook: {
|
||||
id: 'ab-stored-path',
|
||||
title: 'The Valley of Fear',
|
||||
author: 'Arthur Conan Doyle',
|
||||
narrator: null,
|
||||
audibleAsin: null,
|
||||
plexGuid: null,
|
||||
absItemId: null,
|
||||
filePath: storedPath,
|
||||
fileFormat: 'm4b',
|
||||
},
|
||||
downloadHistory: [],
|
||||
});
|
||||
|
||||
const { deleteRequest } = await import('@/lib/services/request-delete.service');
|
||||
const result = await deleteRequest('req-stored-path', 'admin-1');
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.filesDeleted).toBe(true);
|
||||
await expect(fs.access(storedPath)).rejects.toThrow();
|
||||
await expect(fs.readFile(unrelatedFile, 'utf8')).resolves.toBe('user-owned file');
|
||||
expect(configServiceMock.get).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('skips file deletion when a legacy row has no stored path', async () => {
|
||||
const unrelatedPath = path.join(tempRoot, 'Arthur Conan Doyle', 'The Valley of Fear');
|
||||
const unrelatedFile = path.join(unrelatedPath, 'my own rip.m4b');
|
||||
|
||||
await fs.mkdir(unrelatedPath, { recursive: true });
|
||||
await fs.writeFile(unrelatedFile, 'user-owned file');
|
||||
|
||||
prismaMock.request.findFirst.mockResolvedValue({
|
||||
id: 'req-no-stored-path',
|
||||
type: 'audiobook',
|
||||
audiobook: {
|
||||
id: 'ab-no-stored-path',
|
||||
title: 'The Valley of Fear',
|
||||
author: 'Arthur Conan Doyle',
|
||||
narrator: null,
|
||||
audibleAsin: null,
|
||||
plexGuid: null,
|
||||
absItemId: null,
|
||||
filePath: null,
|
||||
fileFormat: null,
|
||||
},
|
||||
downloadHistory: [],
|
||||
});
|
||||
|
||||
const { deleteRequest } = await import('@/lib/services/request-delete.service');
|
||||
const result = await deleteRequest('req-no-stored-path', 'admin-1');
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.filesDeleted).toBe(false);
|
||||
await expect(fs.readFile(unrelatedFile, 'utf8')).resolves.toBe('user-owned file');
|
||||
expect(configServiceMock.get).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user