Compare commits

...

4 Commits

Author SHA1 Message Date
kikootwo bc371860d0 Fix search result limiting after ranking (#277) 2026-08-18 17:26:27 -04:00
kikootwo ae75c914c9 fix: delete requests using stored media path (#276) 2026-08-18 17:26:02 -04:00
kikootwo 425129e8c9 fix: try next release after transient grab failure (#275) 2026-08-18 16:09:36 -04:00
kikootwo a013e2191f fix: prevent retry backlog starvation (#274) 2026-08-18 15:07:10 -04:00
33 changed files with 987 additions and 199 deletions
+26
View File
@@ -0,0 +1,26 @@
# ReadMeABook Agent Instructions
These are the repository-wide instructions for all agents. Also follow the project standards and approval workflow in `CLAUDE.md`.
## Shared Developer Docker Compose
- The canonical developer Compose file is `C:\GIT\ReadMeABook\docker-compose.yml`.
- It is intentionally local-only, marked `skip-worktree`, and contains private machine configuration. Never print its full contents, expose its credentials, or include it in a commit.
- The active data bind mounts are hard-coded to `C:\GIT\ReadMeABook\config`, `cache`, `bookdrop`, `pgdata`, and `redis`, so every worktree uses the same developer data. Keep these mounts absolute.
- From any ReadMeABook worktree, pass its Git root as `--project-directory`. This keeps `build.context: .` pointed at that worktree's source code while the absolute data mounts continue to use the canonical root directories:
```powershell
$rmabWorktree = (git rev-parse --show-toplevel).Trim()
docker compose --project-directory $rmabWorktree -f C:\GIT\ReadMeABook\docker-compose.yml build readmeabook
docker compose --project-directory $rmabWorktree -f C:\GIT\ReadMeABook\docker-compose.yml up -d
```
- To build and start in one command:
```powershell
$rmabWorktree = (git rev-parse --show-toplevel).Trim()
docker compose --project-directory $rmabWorktree -f C:\GIT\ReadMeABook\docker-compose.yml up -d --build
```
- Do not use a plain `docker compose build readmeabook` unless the active worktree's Compose file has first been verified to contain a `build` section. The tracked production Compose normally references a prebuilt registry image, making that command a no-op.
- The developer Compose uses the fixed container name `readmeabook-test`. Before starting it, inspect any existing container with that name. Do not remove or replace an existing container without user approval.
@@ -88,8 +88,10 @@ model Request {
```
3. **Delete Media Files**
- Path: `[media_dir]/[author]/[title]/`
- Path: persisted `Audiobook.filePath` recorded during file organization
- Never rebuilds the path from the current `media_dir` or path template
- **ONLY deletes title folder** (not author folder)
- Missing stored path: skips file deletion and returns `filesDeleted: false`
- Handles missing folders gracefully
4. **Delete from Library Backend**
@@ -214,6 +216,8 @@ where: {
13. ✅ **Plex deletion not enabled in settings** - Log error, continue with soft delete
14. ✅ **Title mismatch in plex_library** - ASIN-based deletion handles title variations (e.g., "(Unabridged)" suffix)
15. ✅ **No ASIN available** - Falls back to exact title/author matching
16. ✅ **Path template changed after organization** - Deletes only the persisted organized path
17. ✅ **Legacy row has no stored path** - Skips file deletion without guessing a path
## Fixed Issues ✅
@@ -224,6 +228,12 @@ where: {
- **Fix:** Changed plex_library deletion to use ASIN-based matching (same as availability check)
- **Result:** Books immediately show as NOT available after deletion, can be re-requested right away
**2. Changed Path Template Deletes an Unrelated Directory**
- **Issue:** Deletion rebuilt a directory from the current media path and template
- **Cause:** `Audiobook.filePath` was persisted during organization but omitted from the deletion query
- **Fix:** Delete only the persisted path; legacy rows without one skip file cleanup
- **Result:** Configuration changes cannot redirect deletion to a newly rendered directory
## File Structure
```
@@ -250,7 +260,6 @@ Queries Updated (deletedAt: null filters):
**No new config required** - uses existing:
- `prowlarr_indexers` (seeding time per indexer)
- `media_dir` (file deletion path)
## Security
@@ -258,6 +267,7 @@ Queries Updated (deletedAt: null filters):
- **Audit Trail:** `deletedBy` tracks admin user ID
- **Soft Delete:** Preserves history, prevents permanent data loss
- **Confirmation Required:** Prevents accidental deletion
- **Stored Path Only:** File cleanup targets the path recorded by the organizer; no template-based fallback
## Monitoring & Logging
+8
View File
@@ -33,6 +33,14 @@ Manages background job queue using Bull (Redis-backed) for async tasks: searchin
5. **plex_recently_added_check** - Lightweight polling of recently added items, match all non-terminal requests
6. **match_plex** - Fuzzy match to Plex item (deprecated - now handled by scan_plex)
## Download Grab Fallback
- Automatic audiobook search passes every above-threshold release to `download_torrent` in ranked order.
- HTTP `429` and `5xx` responses while fetching a `.torrent`/`.nzb` from Prowlarr are source-grab failures, not download-client failures.
- `download_torrent` immediately tries the next ranked release; the successful candidate is the only one written to download history.
- If every ranked grab is temporarily unavailable, the job remains retryable under Bull's exponential backoff. The global failed handler marks the request failed only after all job attempts are exhausted.
- Manual selections carry no alternates and retain their explicit-release behavior.
## Special Behaviors
**monitor_download:**
+3 -3
View File
@@ -19,11 +19,11 @@ Manages recurring/scheduled jobs providing automated tasks (Plex scans, Audible
1. **plex_library_scan** - Default: every 6 hours, full library scan, disabled by default (enable after setup)
2. **plex_recently_added_check** - Default: every 5 minutes, lightweight polling of top 10 recently added items, enabled by default
3. **audible_refresh** - Default: daily midnight, fetches 200 popular + 200 new releases, stores with rankings, disabled by default
4. **retry_missing_torrents** - Default: daily midnight, processes union of `awaiting_search` `awaiting_release` (limit 50), handles both audiobook and ebook requests. Bidirectional transitions: `awaiting_search``awaiting_release` when release date is future + `indexer.skip_unreleased` ON; `awaiting_release``awaiting_search` + run search when release date has passed or setting OFF. Sole owner of these transitions. Enabled by default.
5. **retry_failed_imports** - Default: every 6 hours, re-attempts 'awaiting_import' status (limit 50), enabled by default
4. **retry_missing_torrents** - Default: daily midnight, processes union of `awaiting_search` eligible `awaiting_release` (limit 50), handles both audiobook and ebook requests. Fair ordering: never-searched first, then least-recently searched; future `awaiting_release` rows do not consume retry slots. Bidirectional transitions: `awaiting_search``awaiting_release` when release date is future + `indexer.skip_unreleased` ON; `awaiting_release``awaiting_search` + run search when release date has passed or setting OFF. Sole owner of these transitions. Enabled by default.
5. **retry_failed_imports** - Default: every 6 hours, re-attempts `awaiting_import` status (limit 50), ordered by oldest `lastImportAt` so the backlog rotates fairly; enabled by default
6. **find_missing_ebooks** - Default: daily midnight, scans `downloaded` `available` audiobook requests (limit 50) for missing ebook companions and triggers the existing ebook fetch flow (`addSearchEbookJob`). Gated by `ebook_auto_grab_enabled` AND at least one ebook source enabled (`ebook_annas_archive_enabled` or `ebook_indexer_search_enabled`; legacy `ebook_sidecar_enabled` accepted as Anna's fallback). Skips ebook children in-flight (`pending`, `awaiting_approval`, `searching`, `downloading`, `processing`, `awaiting_search`, `awaiting_release`) or `cancelled`. Retries `failed`/`warn` children up to **5 lifetime auto-retries** per audiobook, tracked in `Request.ebookAutoRetryCount` (nullable; processor-private — manual "Fetch Ebook" never reads/writes it). Per-candidate writes are wrapped in `prisma.$transaction` for race-safety with concurrent auto-grab; counter rolls back if `addSearchEbookJob` throws. Enabled by default. Returns `{ scanned, gapsFound, triggered, created, retried, skippedInFlight, skippedCancelled, skippedCapHit }`.
7. **cleanup_seeded_torrents** - Default: every 30 mins, deletes torrents after seeding requirements met. Respects per-indexer `seedingTimeMinutes` AND `ratioLimit` (BOTH required when set; `0` disables that criterion; both `0` = never cleaned up). Undefined ratio with `ratioLimit > 0` = not met (safe-deny). Enabled by default.
8. **monitor_rss_feeds** - Default: every 15 mins, checks RSS feeds from enabled indexers, matches against `awaiting_search` requests (audiobook and ebook, limit 100). Query is unchanged — release-date gate is applied AFTER a match is found: if matched book is unreleased + `indexer.skip_unreleased` ON, the match is skipped and request status is NOT mutated (retry job owns transitions). Enabled by default.
8. **monitor_rss_feeds** - Default: every 15 mins, checks RSS feeds from enabled indexers, matches against all `awaiting_search` requests (audiobook and ebook) in deterministic pages of 100. Release-date gate is applied AFTER a match is found: if matched book is unreleased + `indexer.skip_unreleased` ON, the match is skipped and request status is NOT mutated (retry job owns transitions). Enabled by default.
## Architecture: Bull + Cron
+2
View File
@@ -20,6 +20,8 @@ Request → search_indexers → rank_results → download_torrent
5. **process_audiobook** - Organize files to media directory
6. **update_plex** - Trigger scan, fuzzy match
Automatic searches keep the ranked candidate list. If Prowlarr returns HTTP `429` or `5xx` while grabbing the selected release, `download_torrent` attributes the error to the indexer/source and falls through to the next release. When all candidates have temporary grab failures, Bull retries the job with backoff.
## Integration Points
**Indexers:** Prowlarr (primary), Jackett (fallback)
+6 -4
View File
@@ -27,7 +27,8 @@ Indexer aggregator for searching multiple torrent/usenet indexers simultaneously
- Minimum score threshold: 50/100
- Filters applied after ranking to remove poor matches
- Ensures at least basic title/author match quality
- maxResults: 100 (increased from 50 for broader search)
- Audiobook searches rank the full deduplicated result set, then retain the best 100
- Interactive responses expose `truncated` and the pre-ranking `rawCount`
**Example:** "Season of Storms" → finds all "Season of Storms" torrents → ranks by author match → filters score < 50
@@ -66,16 +67,17 @@ interface TorrentResult {
**Manual Search** (`POST /api/requests/{id}/manual-search`)
- Triggers automatic search job for requests with status: pending, failed, awaiting_search
- Searches only enabled indexers (title only, maxResults: 100)
- Searches only enabled indexers (title only)
- Ranks all results, filters scores < 50
- Retains the best 100 qualifying results after ranking
- Selects best torrent from filtered results
- Updates request status to 'pending'
**Interactive Search** (`POST /api/requests/{id}/interactive-search`)
- Returns ranked torrent results for user selection
- Searches only enabled indexers (title only or custom, maxResults: 100)
- Searches only enabled indexers (title only or custom)
- Accepts optional custom search title in request body
- Ranks all results, filters scores < 50
- Ranks all results, then returns the best 100 with truncation metadata
- Shows table with: rank, title, size, quality score, seeders, indexer, publish date
- Editable title field allows search refinement
- Available for same statuses as manual search
+7 -1
View File
@@ -247,7 +247,13 @@ type TorrentState =
- `pausedUP``seeding` (unified) / `completed` (legacy) — triggers completion in monitor
- `stoppedUP``seeding` (unified) / `completed` (legacy) — same fix for qBittorrent v5.x
- `pausedDL`/`stoppedDL` remain `paused` — download phase genuinely paused
- Key insight: any `*UP` state is post-download; any `*DL` state is pre-completion
- Key insight: any `*UP` state is post-download; any `*DL` state is pre-completion
**17. Prowlarr grab errors blamed on qBittorrent** - Source HTTP errors were collapsed into `Failed to add torrent to qBittorrent`, so a rate-limited indexer permanently failed the request. Fixed by:
- Forwarding Prowlarr source headers when fetching `.torrent` files
- Preserving the upstream HTTP status as a structured source-grab error
- Trying the remaining ranked releases on HTTP `429`/`5xx`
- Reporting the indexer, release, and upstream status instead of blaming a healthy qBittorrent instance
## Tech Stack
@@ -16,6 +16,7 @@ import { z } from 'zod';
import { RMABLogger } from '@/lib/utils/logger';
const logger = RMABLogger.create('API.AudiobookSearch');
const MAX_RANKED_RESULTS = 100;
const SearchSchema = z.object({
title: z.string(),
@@ -98,7 +99,6 @@ export async function POST(request: NextRequest) {
const groupResults = await prowlarr.searchWithVariations(title, author, {
categories: group.categories,
indexerIds: group.indexerIds,
maxResults: 100, // Limit per group
});
logger.debug(`Group ${i + 1} returned ${groupResults.length} results`);
@@ -116,6 +116,8 @@ export async function POST(request: NextRequest) {
return NextResponse.json({
success: true,
results: [],
truncated: false,
rawCount: 0,
message: 'No torrents/nzbs found',
});
}
@@ -167,6 +169,14 @@ export async function POST(request: NextRequest) {
// User can see scores and make their own decision
logger.debug(`Ranked ${rankedResults.length} results (no threshold filter - user decides)`);
const truncated = rankedResults.length > MAX_RANKED_RESULTS;
const limitedRankedResults = rankedResults.slice(0, MAX_RANKED_RESULTS);
if (truncated) {
logger.info(`Audiobook search truncated after ranking: returning top ${MAX_RANKED_RESULTS} of ${rankedResults.length} ranked results`, {
rawCount: results.length,
});
}
// Log top 3 results with detailed score breakdown for debugging
const top3 = rankedResults.slice(0, 3);
if (top3.length > 0) {
@@ -198,7 +208,7 @@ export async function POST(request: NextRequest) {
}
// Add rank position to each result
const resultsWithRank = rankedResults.map((result, index) => ({
const resultsWithRank = limitedRankedResults.map((result, index) => ({
...result,
rank: index + 1,
}));
@@ -206,8 +216,10 @@ export async function POST(request: NextRequest) {
return NextResponse.json({
success: true,
results: resultsWithRank,
message: rankedResults.length > 0
? `Found ${rankedResults.length} results`
truncated,
rawCount: results.length,
message: limitedRankedResults.length > 0
? `Found ${limitedRankedResults.length} results`
: 'No results found',
});
} catch (error) {
@@ -15,6 +15,7 @@ import { RMABLogger } from '@/lib/utils/logger';
import { resolveInteractiveSearchAccess } from '@/lib/utils/permissions';
const logger = RMABLogger.create('API.InteractiveSearch');
const MAX_RANKED_RESULTS = 100;
/**
* POST /api/requests/[id]/interactive-search
@@ -151,7 +152,6 @@ export async function POST(
const groupResults = await prowlarr.searchWithVariations(searchTitle, searchAuthor, {
categories: group.categories,
indexerIds: group.indexerIds,
maxResults: 100,
});
logger.debug(`Group ${i + 1} returned ${groupResults.length} results`);
@@ -169,6 +169,8 @@ export async function POST(
return NextResponse.json({
success: true,
results: [],
truncated: false,
rawCount: 0,
message: 'No torrents/nzbs found',
});
}
@@ -214,6 +216,14 @@ export async function POST(
// User can see scores and make their own decision
logger.debug(`Ranked ${rankedResults.length} results (no threshold filter - user decides)`);
const truncated = rankedResults.length > MAX_RANKED_RESULTS;
const limitedRankedResults = rankedResults.slice(0, MAX_RANKED_RESULTS);
if (truncated) {
logger.info(`Interactive search truncated after ranking: returning top ${MAX_RANKED_RESULTS} of ${rankedResults.length} ranked results`, {
rawCount: results.length,
});
}
// Log top 3 results with detailed score breakdown for debugging
const top3 = rankedResults.slice(0, 3);
if (top3.length > 0) {
@@ -245,7 +255,7 @@ export async function POST(
}
// Add rank position to each result
const resultsWithRank = rankedResults.map((result, index) => ({
const resultsWithRank = limitedRankedResults.map((result, index) => ({
...result,
rank: index + 1,
}));
@@ -253,8 +263,10 @@ export async function POST(
return NextResponse.json({
success: true,
results: resultsWithRank,
message: rankedResults.length > 0
? `Found ${rankedResults.length} results`
truncated,
rawCount: results.length,
message: limitedRankedResults.length > 0
? `Found ${limitedRankedResults.length} results`
: 'No results found',
});
} catch (error) {
+22 -4
View File
@@ -14,6 +14,7 @@ import { PathMapper, PathMappingConfig } from '../utils/path-mapper';
import {
IDownloadClient, DownloadClientType, ProtocolType,
DownloadInfo, DownloadStatus, AddDownloadOptions, ConnectionTestResult,
DownloadSourceError,
} from '../interfaces/download-client.interface';
const parseTorrent = (parseTorrentModule as any).default || parseTorrentModule;
@@ -193,7 +194,7 @@ export class DelugeService implements IDownloadClient {
torrentResponse = await axios.get(torrentUrl, {
responseType: 'arraybuffer', maxRedirects: 0,
validateStatus: (s) => s >= 200 && s < 300, timeout: DOWNLOAD_CLIENT_TIMEOUT,
headers: { 'User-Agent': RMAB_USER_AGENT },
headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders },
});
if (torrentResponse.data.length > 0) {
const magnetMatch = torrentResponse.data.toString().match(/^magnet:\?[^\s]+$/);
@@ -206,10 +207,27 @@ export class DelugeService implements IDownloadClient {
const loc = error.response.headers['location'];
if (loc?.startsWith('magnet:')) return this.addMagnetLink(loc, category, options);
if (loc?.startsWith('http://') || loc?.startsWith('https://')) {
try { torrentResponse = await axios.get(loc, { responseType: 'arraybuffer', timeout: DOWNLOAD_CLIENT_TIMEOUT, maxRedirects: 5, headers: { 'User-Agent': RMAB_USER_AGENT } }); }
catch { throw new Error('Failed to download torrent file after redirect'); }
try { torrentResponse = await axios.get(loc, { responseType: 'arraybuffer', timeout: DOWNLOAD_CLIENT_TIMEOUT, maxRedirects: 5, headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders } }); }
catch (redirectError) {
if (axios.isAxiosError(redirectError) && redirectError.response?.status) {
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${redirectError.response.status}`,
redirectError.response.status,
loc,
redirectError
);
}
throw new Error('Failed to download torrent file after redirect');
}
} else { throw new Error(`Invalid redirect location: ${loc}`); }
} else { throw new Error(`Failed to download torrent: HTTP ${status}`); }
} else {
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${status}`,
status,
torrentUrl,
error
);
}
}
const torrentBuffer = Buffer.from(torrentResponse.data);
+7 -1
View File
@@ -17,6 +17,7 @@ import {
DownloadStatus,
AddDownloadOptions,
ConnectionTestResult,
DownloadSourceError,
} from '../interfaces/download-client.interface';
const logger = RMABLogger.create('NZBGet');
@@ -255,7 +256,12 @@ export class NZBGetService implements IDownloadClient {
if (axios.isAxiosError(error)) {
const status = error.response?.status;
if (status) {
throw new Error(`Failed to download NZB file: HTTP ${status} from source URL`);
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${status}`,
status,
url,
error
);
}
if (error.code === 'ECONNREFUSED') {
throw new Error('Failed to download NZB file: Connection refused. Is Prowlarr running?');
+30 -1
View File
@@ -20,6 +20,7 @@ import {
DownloadStatus,
AddDownloadOptions,
ConnectionTestResult,
DownloadSourceError,
} from '../interfaces/download-client.interface';
// Handle both ESM and CommonJS imports
@@ -33,6 +34,7 @@ export interface AddTorrentOptions {
tags?: string[];
paused?: boolean;
skipChecking?: boolean;
sourceHeaders?: Record<string, string>;
}
export interface TorrentInfo {
@@ -282,6 +284,11 @@ export class QBittorrentService implements IDownloadClient {
return await this.addTorrentFile(url, category, options);
}
} catch (error) {
// Preserve source/indexer failures so the processor can try another release.
if (error instanceof DownloadSourceError) {
throw error;
}
// Try re-authenticating once if we get a 403 — only meaningful when credentials are configured.
// In auth-optional mode a 403 means the server actually wants auth (e.g. IP no longer whitelisted),
// so retrying login is pointless and would mask the real error.
@@ -378,6 +385,10 @@ export class QBittorrentService implements IDownloadClient {
maxRedirects: 0,
validateStatus: (status) => status >= 200 && status < 300, // Only 2xx is success
timeout: DOWNLOAD_CLIENT_TIMEOUT,
headers: {
'User-Agent': RMAB_USER_AGENT,
...options?.sourceHeaders,
},
});
logger.info(` Got 2xx response, size=${torrentResponse.data.length} bytes`);
@@ -421,10 +432,22 @@ export class QBittorrentService implements IDownloadClient {
responseType: 'arraybuffer',
timeout: DOWNLOAD_CLIENT_TIMEOUT,
maxRedirects: 5,
headers: {
'User-Agent': RMAB_USER_AGENT,
...options?.sourceHeaders,
},
});
logger.info(` After following redirect: size=${torrentResponse.data.length} bytes`);
} catch (redirectError) {
logger.error('Failed to follow redirect', { error: redirectError instanceof Error ? redirectError.message : String(redirectError) });
if (axios.isAxiosError(redirectError) && redirectError.response?.status) {
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${redirectError.response.status}`,
redirectError.response.status,
location,
redirectError
);
}
throw new Error('Failed to download torrent file after redirect');
}
} else {
@@ -433,7 +456,12 @@ export class QBittorrentService implements IDownloadClient {
} else {
// Non-redirect error (4xx, 5xx)
logger.error(`HTTP error ${status}`, { error: error.message });
throw new Error(`Failed to download torrent: HTTP ${status}`);
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${status}`,
status,
torrentUrl,
error
);
}
}
@@ -1077,6 +1105,7 @@ export class QBittorrentService implements IDownloadClient {
category: options?.category,
paused: options?.paused,
tags: ['audiobook'],
sourceHeaders: options?.sourceHeaders,
});
}
+7 -1
View File
@@ -17,6 +17,7 @@ import {
DownloadStatus,
AddDownloadOptions,
ConnectionTestResult,
DownloadSourceError,
} from '../interfaces/download-client.interface';
const logger = RMABLogger.create('SABnzbd');
@@ -519,7 +520,12 @@ export class SABnzbdService implements IDownloadClient {
if (axios.isAxiosError(error)) {
const status = error.response?.status;
if (status) {
throw new Error(`Failed to download NZB file: HTTP ${status} from source URL`);
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${status}`,
status,
url,
error
);
}
if (error.code === 'ECONNREFUSED') {
throw new Error('Failed to download NZB file: Connection refused. Is Prowlarr running?');
+18 -4
View File
@@ -19,6 +19,7 @@ import {
DownloadStatus,
AddDownloadOptions,
ConnectionTestResult,
DownloadSourceError,
} from '../interfaces/download-client.interface';
// Handle both ESM and CommonJS imports
@@ -278,7 +279,7 @@ export class TransmissionService implements IDownloadClient {
maxRedirects: 0,
validateStatus: (status) => status >= 200 && status < 300,
timeout: DOWNLOAD_CLIENT_TIMEOUT,
headers: { 'User-Agent': RMAB_USER_AGENT },
headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders },
});
// Check if response body is a magnet link
@@ -308,16 +309,29 @@ export class TransmissionService implements IDownloadClient {
responseType: 'arraybuffer',
timeout: DOWNLOAD_CLIENT_TIMEOUT,
maxRedirects: 5,
headers: { 'User-Agent': RMAB_USER_AGENT },
headers: { 'User-Agent': RMAB_USER_AGENT, ...options?.sourceHeaders },
});
} catch {
} catch (redirectError) {
if (axios.isAxiosError(redirectError) && redirectError.response?.status) {
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${redirectError.response.status}`,
redirectError.response.status,
location,
redirectError
);
}
throw new Error('Failed to download torrent file after redirect');
}
} else {
throw new Error(`Invalid redirect location: ${location}`);
}
} else {
throw new Error(`Failed to download torrent: HTTP ${status}`);
throw new DownloadSourceError(
`Grab failed: source returned HTTP ${status}`,
status,
torrentUrl,
error
);
}
}
@@ -106,6 +106,33 @@ export interface AddDownloadOptions {
sourceHeaders?: Record<string, string>;
}
/**
* Failure while fetching a .torrent or .nzb from its source URL.
* Keeping this distinct from download-client API failures lets the pipeline
* try another release without blaming a healthy download client.
*/
export class DownloadSourceError extends Error {
readonly name = 'DownloadSourceError';
constructor(
message: string,
public readonly status: number | undefined,
public readonly sourceUrl: string,
public readonly originalError?: unknown
) {
super(message);
}
}
/** HTTP source failures that may clear on their own or succeed via another indexer. */
export function isRetryableDownloadSourceError(
error: unknown
): error is DownloadSourceError {
return error instanceof DownloadSourceError
&& typeof error.status === 'number'
&& (error.status === 429 || (error.status >= 500 && error.status < 600));
}
/** Result of a connection test */
export interface ConnectionTestResult {
success: boolean;
+136 -86
View File
@@ -10,6 +10,10 @@ import { getDownloadClientManager } from '../services/download-client-manager.se
import { ProwlarrService } from '../integrations/prowlarr.service';
import { RMABLogger } from '../utils/logger';
import { isTransientConnectionError } from '../utils/connection-errors';
import {
DownloadSourceError,
isRetryableDownloadSourceError,
} from '../interfaces/download-client.interface';
/**
* Process download job
@@ -17,7 +21,8 @@ import { isTransientConnectionError } from '../utils/connection-errors';
* Adds selected result to download client and starts monitoring
*/
export async function processDownloadTorrent(payload: DownloadTorrentPayload): Promise<any> {
const { requestId, audiobook, torrent, jobId } = payload;
const { requestId, audiobook, torrent, alternateTorrents = [], jobId } = payload;
const candidates = [torrent, ...alternateTorrents];
const logger = RMABLogger.forJob(jobId, 'DownloadTorrent');
@@ -27,6 +32,7 @@ export async function processDownloadTorrent(payload: DownloadTorrentPayload): P
seeders: torrent.seeders,
format: torrent.format,
indexer: torrent.indexer,
alternateCount: alternateTorrents.length,
});
try {
@@ -43,24 +49,9 @@ export async function processDownloadTorrent(payload: DownloadTorrentPayload): P
},
});
// Detect protocol from result and get appropriate client
const isUsenet = ProwlarrService.isNZBResult(torrent);
const protocol = isUsenet ? 'usenet' : 'torrent';
const config = await getConfigService();
const manager = getDownloadClientManager(config);
const client = await manager.getClientServiceForProtocol(protocol);
if (!client) {
throw new Error(`No ${protocol} download client configured. Please add a ${protocol} client in Settings > Download Clients.`);
}
// Get client config for category
const clientConfig = await manager.getClientForProtocol(protocol);
const category = clientConfig?.category || 'readmeabook';
logger.info(`Routing to ${client.clientType} (${client.protocol})`);
// Include Prowlarr API key as source header so NZB/torrent downloads from
// Prowlarr proxy URLs are authenticated (fixes 403 for indexers like NZBFinder)
const prowlarrApiKey = (await config.getMany(['prowlarr_api_key'])).prowlarr_api_key || process.env.PROWLARR_API_KEY;
@@ -69,90 +60,149 @@ export async function processDownloadTorrent(payload: DownloadTorrentPayload): P
sourceHeaders['X-Api-Key'] = prowlarrApiKey;
}
// Add download via unified interface
const downloadClientId = await client.addDownload(torrent.downloadUrl, {
category,
priority: 'normal',
sourceHeaders,
});
for (let candidateIndex = 0; candidateIndex < candidates.length; candidateIndex++) {
const candidate = candidates[candidateIndex];
const isUsenet = ProwlarrService.isNZBResult(candidate);
const protocol = isUsenet ? 'usenet' : 'torrent';
const client = await manager.getClientServiceForProtocol(protocol);
logger.info(`Download added with ID: ${downloadClientId}`);
if (!client) {
throw new Error(`No ${protocol} download client configured. Please add a ${protocol} client in Settings > Download Clients.`);
}
// Create DownloadHistory record
// Determine indexer page URL - exclude magnet links from guid fallback
const indexerPageUrl = torrent.infoUrl || (torrent.guid?.startsWith('magnet:') ? null : torrent.guid);
const clientConfig = await manager.getClientForProtocol(protocol);
const category = clientConfig?.category || 'readmeabook';
const downloadHistory = await prisma.downloadHistory.create({
data: {
logger.info(
`Trying release ${candidateIndex + 1}/${candidates.length}: ${candidate.title}`,
{ indexer: candidate.indexer, protocol, downloadClient: client.clientType }
);
let downloadClientId: string;
try {
downloadClientId = await client.addDownload(candidate.downloadUrl, {
category,
priority: 'normal',
sourceHeaders,
});
} catch (error) {
if (!(error instanceof DownloadSourceError)) {
throw error;
}
const contextualError = new DownloadSourceError(
`Grab failed: ${candidate.indexer} returned HTTP ${error.status} while fetching "${candidate.title}" through Prowlarr`,
error.status,
error.sourceUrl,
error
);
if (!isRetryableDownloadSourceError(error)) {
throw contextualError;
}
const hasNextCandidate = candidateIndex < candidates.length - 1;
logger.warn(
`${contextualError.message}${hasNextCandidate ? '; trying next ranked release' : ''}`,
{
indexer: candidate.indexer,
upstreamStatus: error.status,
candidate: candidateIndex + 1,
candidateCount: candidates.length,
}
);
if (hasNextCandidate) {
continue;
}
throw contextualError;
}
logger.info(`Download added with ID: ${downloadClientId}`);
// Create DownloadHistory record. Exclude magnet links from the indexer-page fallback.
const indexerPageUrl = candidate.infoUrl || (candidate.guid?.startsWith('magnet:') ? null : candidate.guid);
const downloadHistory = await prisma.downloadHistory.create({
data: {
requestId,
indexerName: candidate.indexer,
indexerId: candidate.indexerId,
downloadClient: client.clientType,
downloadClientId,
torrentName: candidate.title,
// Set protocol-specific ID fields for backward compatibility
torrentHash: client.protocol === 'torrent' ? (candidate.infoHash || downloadClientId) : undefined,
nzbId: client.protocol === 'usenet' ? downloadClientId : undefined,
torrentSizeBytes: candidate.size,
torrentUrl: indexerPageUrl,
magnetLink: candidate.downloadUrl,
seeders: candidate.seeders || 0,
leechers: candidate.leechers || 0,
downloadStatus: 'downloading',
selected: true,
startedAt: new Date(),
},
});
logger.info(`Created download history record: ${downloadHistory.id}`);
// Send grab notification (non-blocking — failures here don't fail the download)
const jobQueue = getJobQueueService();
const grabMessage = `${candidate.title} via ${candidate.indexer} (${client.clientType})`;
await jobQueue.addNotificationJob(
'request_grabbed',
requestId,
indexerName: torrent.indexer,
indexerId: torrent.indexerId,
downloadClient: client.clientType,
audiobook.title,
audiobook.author,
request.user.plexUsername || 'Unknown User',
grabMessage,
request.type
).catch((error) => {
logger.error('Failed to queue grab notification', { error: error instanceof Error ? error.message : String(error) });
});
// Trigger monitor download job with initial delay
await jobQueue.addMonitorJob(
requestId,
downloadHistory.id,
downloadClientId,
torrentName: torrent.title,
// Set protocol-specific ID fields for backward compatibility
torrentHash: client.protocol === 'torrent' ? (torrent.infoHash || downloadClientId) : undefined,
nzbId: client.protocol === 'usenet' ? downloadClientId : undefined,
torrentSizeBytes: torrent.size,
torrentUrl: indexerPageUrl,
magnetLink: torrent.downloadUrl,
seeders: torrent.seeders || 0,
leechers: torrent.leechers || 0,
downloadStatus: 'downloading',
selected: true,
startedAt: new Date(),
},
});
client.clientType,
3 // Wait 3 seconds before first check
);
logger.info(`Created download history record: ${downloadHistory.id}`);
logger.info(`Started monitoring job for request ${requestId} (${client.clientType}, 3s initial delay)`);
// Send grab notification (non-blocking — failures here don't fail the download)
const jobQueue = getJobQueueService();
const grabMessage = `${torrent.title} via ${torrent.indexer} (${client.clientType})`;
await jobQueue.addNotificationJob(
'request_grabbed',
requestId,
audiobook.title,
audiobook.author,
request.user.plexUsername || 'Unknown User',
grabMessage,
request.type
).catch((error) => {
logger.error('Failed to queue grab notification', { error: error instanceof Error ? error.message : String(error) });
});
return {
success: true,
message: `Download added to ${client.clientType} and monitoring started`,
requestId,
downloadHistoryId: downloadHistory.id,
downloadClientId,
torrent: {
title: candidate.title,
size: candidate.size,
seeders: candidate.seeders || 0,
format: candidate.format,
},
};
}
// Trigger monitor download job with initial delay
await jobQueue.addMonitorJob(
requestId,
downloadHistory.id,
downloadClientId,
client.clientType,
3 // Wait 3 seconds before first check
);
logger.info(`Started monitoring job for request ${requestId} (${client.clientType}, 3s initial delay)`);
return {
success: true,
message: `Download added to ${client.clientType} and monitoring started`,
requestId,
downloadHistoryId: downloadHistory.id,
downloadClientId,
torrent: {
title: torrent.title,
size: torrent.size,
seeders: torrent.seeders || 0,
format: torrent.format,
},
};
throw new Error('No release candidates were available to download');
} catch (error) {
logger.error(`Error: ${error instanceof Error ? error.message : 'Unknown error'}`);
if (isTransientConnectionError(error)) {
if (isTransientConnectionError(error) || isRetryableDownloadSourceError(error)) {
// Connection error — don't mark request as failed yet.
// Bull will retry this job (3 attempts with exponential backoff).
// If all retries are exhausted, the global failed handler marks it failed.
logger.warn(`Download client unreachable for request ${requestId}, allowing Bull to retry`);
logger.warn(
isRetryableDownloadSourceError(error)
? `All ranked release grabs were temporarily unavailable for request ${requestId}, allowing Bull to retry`
: `Download client unreachable for request ${requestId}, allowing Bull to retry`
);
} else {
// Permanent error — mark request as failed immediately
await prisma.request.update({
@@ -17,6 +17,8 @@ export interface MonitorRssFeedsPayload {
scheduledJobId?: string;
}
const REQUEST_PAGE_SIZE = 100;
export async function processMonitorRssFeeds(payload: MonitorRssFeedsPayload): Promise<any> {
const { jobId, scheduledJobId } = payload;
const logger = RMABLogger.forJob(jobId, 'MonitorRssFeeds');
@@ -63,16 +65,35 @@ export async function processMonitorRssFeeds(payload: MonitorRssFeedsPayload): P
return { success: true, message: 'No RSS results', matched: 0 };
}
// Get all active requests awaiting search (audiobooks and ebooks)
// Both types can be matched against RSS torrent feeds
const missingRequests = await prisma.request.findMany({
// Get every active request awaiting search (audiobooks and ebooks) in
// deterministic pages. A single unordered take(100) repeatedly inspected
// the same rows and made requests beyond that batch invisible to RSS.
const firstPage = await prisma.request.findMany({
where: {
status: 'awaiting_search',
deletedAt: null,
},
include: { audiobook: true },
take: 100,
orderBy: { id: 'asc' },
take: REQUEST_PAGE_SIZE,
});
const missingRequests = [...firstPage];
let page = firstPage;
while (page.length === REQUEST_PAGE_SIZE) {
page = await prisma.request.findMany({
where: {
status: 'awaiting_search',
deletedAt: null,
},
include: { audiobook: true },
orderBy: { id: 'asc' },
cursor: { id: page[page.length - 1].id },
skip: 1,
take: REQUEST_PAGE_SIZE,
});
missingRequests.push(...page);
}
logger.info(`Found ${missingRequests.length} requests awaiting search`);
@@ -61,6 +61,11 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
take: 1,
},
},
orderBy: [
{ lastImportAt: { sort: 'asc', nulls: 'first' } },
{ createdAt: 'asc' },
{ id: 'asc' },
],
take: 50, // Limit to 50 requests per run
});
@@ -81,6 +86,13 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
for (const request of requests) {
try {
// Claim this retry slot immediately so every selected row rotates to
// the back of the next batch, including malformed rows we must skip.
await prisma.request.update({
where: { id: request.id },
data: { lastImportAt: new Date() },
});
// Get the download path from the most recent download history
const downloadHistory = request.downloadHistory[0];
@@ -30,15 +30,36 @@ export async function processRetryMissingTorrents(payload: RetryMissingTorrentsP
const configService = getConfigService();
const skipUnreleasedSetting = (await configService.get('indexer.skip_unreleased')) !== 'false';
// Find all active requests in awaiting_search OR awaiting_release status
// Release dates are stored as date-only values. When the release gate is
// enabled, do not let still-future awaiting_release rows consume one of
// the limited retry slots; they become eligible automatically on release day.
const todayUtc = new Date();
todayUtc.setUTCHours(0, 0, 0, 0);
// Process never-searched requests first, then rotate through the least
// recently searched. Without an explicit order, PostgreSQL repeatedly
// returned the same physical first 50 rows and starved the rest forever.
const requests = await prisma.request.findMany({
where: {
status: { in: ['awaiting_search', 'awaiting_release'] },
deletedAt: null,
...(skipUnreleasedSetting
? {
OR: [
{ status: 'awaiting_search' },
{ status: 'awaiting_release', releaseDate: null },
{ status: 'awaiting_release', releaseDate: { lte: todayUtc } },
],
}
: { status: { in: ['awaiting_search', 'awaiting_release'] } }),
},
include: {
audiobook: true,
},
orderBy: [
{ lastSearchAt: { sort: 'asc', nulls: 'first' } },
{ createdAt: 'asc' },
{ id: 'asc' },
],
take: 50,
});
@@ -13,6 +13,8 @@ import { getLanguageForRegion } from '../constants/language-config';
import { filterBlockedResults } from '../utils/filter-blocked-results';
import type { AudibleRegion } from '../types/audible';
const MAX_RANKED_RESULTS = 100;
/**
* Process search indexers job
* Searches configured indexers for audiobook torrents
@@ -103,7 +105,6 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
categories: group.categories,
indexerIds: group.indexerIds,
minSeeders: 1, // Only torrents with at least 1 seeder
maxResults: 100, // Limit per group
});
logger.info(`Group ${i + 1} returned ${groupResults.length} results`);
@@ -201,15 +202,20 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
// Dual threshold filtering:
// 1. Base score must be >= 50 (quality minimum)
// 2. Final score must be >= 50 (not disqualified by negative bonuses)
const filteredResults = rankedResults.filter(result =>
const qualifyingResults = rankedResults.filter(result =>
result.score >= 50 && result.finalScore >= 50
);
const resultsTruncated = qualifyingResults.length > MAX_RANKED_RESULTS;
const filteredResults = qualifyingResults.slice(0, MAX_RANKED_RESULTS);
const disqualifiedByNegativeBonus = rankedResults.filter(result =>
result.score >= 50 && result.finalScore < 50
).length;
logger.info(`Ranked ${rankedResults.length} results, ${filteredResults.length} above threshold (50/100 base + final)`);
logger.info(`Ranked ${rankedResults.length} results, ${qualifyingResults.length} above threshold (50/100 base + final)`);
if (resultsTruncated) {
logger.info(`Limited automatic search to the top ${MAX_RANKED_RESULTS} of ${qualifyingResults.length} qualifying results after ranking`);
}
if (disqualifiedByNegativeBonus > 0) {
logger.info(`${disqualifiedByNegativeBonus} torrents disqualified by negative flag bonuses`);
}
@@ -283,7 +289,7 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
id: audiobook.id,
title: audiobook.title,
author: audiobook.author,
}, bestResult);
}, bestResult, filteredResults.slice(1));
return {
success: true,
+5 -1
View File
@@ -58,6 +58,8 @@ export interface DownloadTorrentPayload extends JobPayload {
author: string;
};
torrent: TorrentResult;
/** Ranked candidates after `torrent`, used when an upstream grab is rate-limited/unavailable. */
alternateTorrents?: TorrentResult[];
}
export interface MonitorDownloadPayload extends JobPayload {
@@ -599,7 +601,8 @@ export class JobQueueService {
async addDownloadJob(
requestId: string,
audiobook: { id: string; title: string; author: string },
torrent: TorrentResult
torrent: TorrentResult,
alternateTorrents: TorrentResult[] = []
): Promise<string> {
return await this.addJob(
'download_torrent',
@@ -607,6 +610,7 @@ export class JobQueueService {
requestId,
audiobook,
torrent,
alternateTorrents,
} as DownloadTorrentPayload,
{
priority: 9, // High priority - download selected torrent
+35 -61
View File
@@ -9,7 +9,6 @@ import { prisma } from '../db';
import * as fs from 'fs/promises';
import * as path from 'path';
import { RMABLogger } from '../utils/logger';
import { buildAudiobookPath } from '../utils/file-organizer';
import { CLIENT_PROTOCOL_MAP, DownloadClientType } from '../interfaces/download-client.interface';
const logger = RMABLogger.create('RequestDelete');
@@ -66,6 +65,7 @@ export async function deleteRequest(
audibleAsin: true,
plexGuid: true,
absItemId: true,
filePath: true,
fileFormat: true,
},
},
@@ -202,72 +202,46 @@ export async function deleteRequest(
// For ebooks: delete only ebook files (leave audiobook files intact)
let filesDeleted = false;
try {
const { getConfigService } = await import('./config.service');
const configService = getConfigService();
const mediaDir = (await configService.get('media_dir')) || '/media/audiobooks';
// Use ebook-specific template for ebook requests, with fallback to audiobook template
const audiobookTemplate = (await configService.get('audiobook_path_template')) || '{author}/{title} {asin}';
const template = isEbook
? (await configService.get('ebook_path_template')) || audiobookTemplate
: audiobookTemplate;
const titleFolderPath = request.audiobook.filePath;
// Fetch year from audible cache if ASIN is available
let year: number | undefined;
if (request.audiobook.audibleAsin) {
const audibleCache = await prisma.audibleCache.findUnique({
where: { asin: request.audiobook.audibleAsin },
select: { releaseDate: true },
});
if (audibleCache?.releaseDate) {
year = new Date(audibleCache.releaseDate).getFullYear();
}
}
if (!titleFolderPath) {
logger.warn(
`Skipping media file deletion for request ${requestId}: no stored file path`
);
} else {
// Check if the directory recorded during organization still exists.
try {
await fs.access(titleFolderPath);
// Build path using centralized function
const titleFolderPath = buildAudiobookPath(
mediaDir,
template,
{
author: request.audiobook.author,
title: request.audiobook.title,
narrator: request.audiobook.narrator || undefined,
asin: request.audiobook.audibleAsin || undefined,
year,
}
);
if (isEbook) {
// For ebooks: only delete ebook files, leave audiobook files intact
const ebookExtensions = ['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.fb2', '.cbz', '.cbr'];
const files = await fs.readdir(titleFolderPath);
// Check if folder exists
try {
await fs.access(titleFolderPath);
if (isEbook) {
// For ebooks: only delete ebook files, leave audiobook files intact
const ebookExtensions = ['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.fb2', '.cbz', '.cbr'];
const files = await fs.readdir(titleFolderPath);
let deletedCount = 0;
for (const file of files) {
const ext = path.extname(file).toLowerCase();
if (ebookExtensions.includes(ext)) {
const filePath = path.join(titleFolderPath, file);
await fs.unlink(filePath);
logger.info(`Deleted ebook file: ${file}`);
deletedCount++;
let deletedCount = 0;
for (const file of files) {
const ext = path.extname(file).toLowerCase();
if (ebookExtensions.includes(ext)) {
const filePath = path.join(titleFolderPath, file);
await fs.unlink(filePath);
logger.info(`Deleted ebook file: ${file}`);
deletedCount++;
}
}
}
filesDeleted = deletedCount > 0;
logger.info(`Deleted ${deletedCount} ebook file(s) from: ${titleFolderPath}`);
} else {
// For audiobooks: delete the entire title folder
await fs.rm(titleFolderPath, { recursive: true, force: true });
logger.info(`Deleted media directory: ${titleFolderPath}`);
filesDeleted = true;
filesDeleted = deletedCount > 0;
logger.info(`Deleted ${deletedCount} ebook file(s) from: ${titleFolderPath}`);
} else {
// For audiobooks: delete the entire title folder
await fs.rm(titleFolderPath, { recursive: true, force: true });
logger.info(`Deleted media directory: ${titleFolderPath}`);
filesDeleted = true;
}
} catch (accessError) {
// Folder doesn't exist - that's okay
logger.info(`Media directory not found: ${titleFolderPath}`);
filesDeleted = false;
}
} catch (accessError) {
// Folder doesn't exist - that's okay
logger.info(`Media directory not found: ${titleFolderPath}`);
filesDeleted = false;
}
} catch (error) {
logger.error(
@@ -20,6 +20,10 @@ const rankTorrentsMock = vi.hoisted(() => vi.fn());
const groupIndexersMock = vi.hoisted(() => vi.fn());
const groupDescriptionMock = vi.hoisted(() => vi.fn(() => 'Group'));
vi.mock('@/lib/db', () => ({
prisma: {},
}));
vi.mock('@/lib/middleware/auth', () => ({
requireAuth: requireAuthMock,
}));
@@ -94,6 +98,45 @@ describe('Audiobooks search torrents route', () => {
expect(payload.results[0].rank).toBe(1);
expect(rankTorrentsMock).toHaveBeenCalled();
});
it('ranks all candidates before returning the top 100 with truncation metadata', async () => {
authRequest.json.mockResolvedValue({ title: 'Title', author: 'Author' });
configServiceMock.get
.mockResolvedValueOnce(JSON.stringify([{ id: 1, name: 'Indexer', protocol: 'torrent', priority: 10 }]))
.mockResolvedValueOnce(null);
groupIndexersMock.mockReturnValue({ groups: [{ categories: [3030], indexerIds: [1] }], skippedIndexers: [] });
const candidates = Array.from({ length: 101 }, (_, index) => ({
title: index === 100 ? 'Best result' : `Result ${index}`,
size: 100,
indexer: 'Indexer',
indexerId: 1,
}));
const rankedCandidates = [candidates[100], ...candidates.slice(0, 100)].map((result, index) => ({
...result,
score: 100 - index,
breakdown: { matchScore: 50, formatScore: 0, sizeScore: 0, seederScore: 0, notes: [] },
bonusPoints: 0,
bonusModifiers: [],
finalScore: 100 - index,
}));
prowlarrMock.searchWithVariations.mockResolvedValue(candidates);
rankTorrentsMock.mockReturnValue(rankedCandidates);
const { POST } = await import('@/app/api/audiobooks/search-torrents/route');
const response = await POST({} as any);
const payload = await response.json();
expect(rankTorrentsMock.mock.calls[0][0]).toHaveLength(101);
expect(prowlarrMock.searchWithVariations).toHaveBeenCalledWith('Title', 'Author', {
categories: [3030],
indexerIds: [1],
});
expect(payload.results).toHaveLength(100);
expect(payload.results[0]).toEqual(expect.objectContaining({ title: 'Best result', rank: 1 }));
expect(payload.truncated).toBe(true);
expect(payload.rawCount).toBe(101);
});
});
+45
View File
@@ -153,6 +153,51 @@ describe('Request action routes', () => {
);
});
it('ranks the complete interactive result set before returning the top 100', async () => {
authRequest.json.mockResolvedValue({});
prismaMock.request.findUnique.mockResolvedValueOnce({
id: 'req-complete-ranking',
userId: 'user-1',
audiobook: { title: 'Title', author: 'Author', audibleAsin: null },
});
prismaMock.user.findUnique.mockResolvedValueOnce({
role: 'user',
interactiveSearchAccess: null,
});
configServiceMock.get.mockResolvedValueOnce(JSON.stringify([{ id: 1, priority: 10, categories: [3030] }]));
configServiceMock.get.mockResolvedValueOnce(null);
groupIndexersMock.mockReturnValue({ groups: [{ categories: [3030], indexerIds: [1] }], skippedIndexers: [] });
const candidates = Array.from({ length: 101 }, (_, index) => ({
title: index === 100 ? 'Best result' : `Result ${index}`,
size: 100,
}));
const rankedCandidates = [candidates[100], ...candidates.slice(0, 100)].map((result, index) => ({
...result,
score: 100 - index,
breakdown: { matchScore: 50, formatScore: 0, sizeScore: 0, seederScore: 0, notes: [] },
bonusPoints: 0,
bonusModifiers: [],
finalScore: 100 - index,
}));
prowlarrMock.searchWithVariations.mockResolvedValueOnce(candidates);
rankTorrentsMock.mockReturnValueOnce(rankedCandidates);
const { POST } = await import('@/app/api/requests/[id]/interactive-search/route');
const response = await POST({} as any, { params: Promise.resolve({ id: 'req-complete-ranking' }) });
const payload = await response.json();
expect(rankTorrentsMock.mock.calls[0][0]).toHaveLength(101);
expect(prowlarrMock.searchWithVariations).toHaveBeenCalledWith('Title', 'Author', {
categories: [3030],
indexerIds: [1],
});
expect(payload.results).toHaveLength(100);
expect(payload.results[0]).toEqual(expect.objectContaining({ title: 'Best result', rank: 1 }));
expect(payload.truncated).toBe(true);
expect(payload.rawCount).toBe(101);
});
it('performs interactive search gracefully when runtime fetch fails', async () => {
authRequest.json.mockResolvedValue({});
prismaMock.request.findUnique.mockResolvedValueOnce({
@@ -6,6 +6,7 @@
import path from 'path';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { QBittorrentService, getQBittorrentService, invalidateQBittorrentService } from '@/lib/integrations/qbittorrent.service';
import { DownloadSourceError } from '@/lib/interfaces/download-client.interface';
const clientMock = vi.hoisted(() => ({
get: vi.fn(),
@@ -693,6 +694,32 @@ describe('QBittorrentService', () => {
);
});
it('preserves upstream HTTP status and forwards source headers', async () => {
const service = new QBittorrentService('http://qb', 'user', 'pass');
(service as any).cookie = 'SID=source-error';
vi.spyOn(service as any, 'ensureCategory').mockResolvedValue(undefined);
axiosMock.get.mockRejectedValueOnce({
isAxiosError: true,
response: { status: 500, headers: {} },
message: 'Request failed',
});
const error = await service.addDownload(
'https://prowlarr/1/download/limited',
{ sourceHeaders: { 'X-Api-Key': 'secret' } }
).catch((caught) => caught);
expect(error).toBeInstanceOf(DownloadSourceError);
expect(error).toMatchObject({ status: 500 });
expect(axiosMock.get).toHaveBeenCalledWith(
'https://prowlarr/1/download/limited',
expect.objectContaining({
headers: expect.objectContaining({ 'X-Api-Key': 'secret' }),
})
);
});
it('throws for invalid redirect locations when fetching torrents', async () => {
const service = new QBittorrentService('http://qb', 'user', 'pass');
@@ -6,6 +6,7 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { createPrismaMock } from '../helpers/prisma';
import { createJobQueueMock } from '../helpers/job-queue';
import { DownloadSourceError } from '@/lib/interfaces/download-client.interface';
const prismaMock = createPrismaMock();
const configMock = vi.hoisted(() => ({
@@ -214,4 +215,107 @@ describe('processDownloadTorrent', () => {
expect(downloadClientManagerMock.getClientServiceForProtocol).toHaveBeenCalledWith('usenet');
});
it('tries the next ranked release when an indexer grab returns 500', async () => {
const fallbackTorrent = {
...torrentPayload.torrent,
indexer: 'Healthy Indexer',
indexerId: 2,
title: 'Book - Author - Fallback',
downloadUrl: 'https://prowlarr/2/download/fallback',
guid: 'guid-fallback',
};
const qbtClientMock = {
clientType: 'qbittorrent',
protocol: 'torrent',
addDownload: vi.fn()
.mockRejectedValueOnce(new DownloadSourceError(
'Grab failed: source returned HTTP 500',
500,
'https://prowlarr/1/download/limited'
))
.mockResolvedValueOnce('fallback-hash'),
};
downloadClientManagerMock.getClientServiceForProtocol.mockResolvedValue(qbtClientMock);
downloadClientManagerMock.getClientForProtocol.mockResolvedValue({
id: 'client-1',
type: 'qbittorrent',
enabled: true,
category: 'readmeabook',
});
prismaMock.request.update.mockResolvedValue({
type: 'audiobook',
user: { plexUsername: 'testuser' },
});
prismaMock.downloadHistory.create.mockResolvedValue({ id: 'dh-fallback' });
const { processDownloadTorrent } = await import('@/lib/processors/download-torrent.processor');
const result = await processDownloadTorrent({
...torrentPayload,
torrent: {
...torrentPayload.torrent,
downloadUrl: 'https://prowlarr/1/download/limited',
},
alternateTorrents: [fallbackTorrent],
});
expect(result.success).toBe(true);
expect(result.torrent.title).toBe(fallbackTorrent.title);
expect(qbtClientMock.addDownload).toHaveBeenNthCalledWith(
1,
'https://prowlarr/1/download/limited',
expect.anything()
);
expect(qbtClientMock.addDownload).toHaveBeenNthCalledWith(
2,
fallbackTorrent.downloadUrl,
expect.anything()
);
expect(prismaMock.downloadHistory.create).toHaveBeenCalledWith(
expect.objectContaining({
data: expect.objectContaining({
indexerName: 'Healthy Indexer',
torrentName: fallbackTorrent.title,
downloadClientId: 'fallback-hash',
}),
})
);
});
it('leaves the request retryable when every ranked grab is temporarily unavailable', async () => {
const qbtClientMock = {
clientType: 'qbittorrent',
protocol: 'torrent',
addDownload: vi.fn().mockRejectedValue(new DownloadSourceError(
'Grab failed: source returned HTTP 429',
429,
'https://prowlarr/1/download/limited'
)),
};
downloadClientManagerMock.getClientServiceForProtocol.mockResolvedValue(qbtClientMock);
downloadClientManagerMock.getClientForProtocol.mockResolvedValue({
id: 'client-1',
type: 'qbittorrent',
enabled: true,
category: 'readmeabook',
});
prismaMock.request.update.mockResolvedValue({
type: 'audiobook',
user: { plexUsername: 'testuser' },
});
const { processDownloadTorrent } = await import('@/lib/processors/download-torrent.processor');
await expect(processDownloadTorrent({
...torrentPayload,
torrent: {
...torrentPayload.torrent,
downloadUrl: 'https://prowlarr/1/download/limited',
},
})).rejects.toThrow('Indexer returned HTTP 429');
expect(prismaMock.request.update).not.toHaveBeenCalledWith(
expect.objectContaining({ data: expect.objectContaining({ status: 'failed' }) })
);
expect(prismaMock.downloadHistory.create).not.toHaveBeenCalled();
});
});
@@ -34,6 +34,7 @@ function futureDate(days = 30): Date {
describe('processMonitorRssFeeds', () => {
beforeEach(() => {
vi.useRealTimers();
vi.clearAllMocks();
// Default to empty blocklist so the filter is a no-op unless a test overrides.
prismaMock.blockedRelease.findMany.mockResolvedValue([]);
@@ -73,6 +74,70 @@ describe('processMonitorRssFeeds', () => {
'req-1',
expect.objectContaining({ title: 'Great Book', author: 'Author Name' })
);
expect(prismaMock.request.findMany).toHaveBeenCalledWith(expect.objectContaining({
orderBy: { id: 'asc' },
take: 100,
}));
});
it('pages beyond the first 100 awaiting requests', async () => {
vi.useFakeTimers();
configMock.get.mockImplementation(async (key: string) => {
if (key === 'prowlarr_indexers') {
return JSON.stringify([{ id: 1, name: 'Indexer', rssEnabled: true }]);
}
return null;
});
prowlarrMock.getAllRssFeeds.mockResolvedValue([
{ title: 'Target Story - Target Author' },
]);
const firstPage = Array.from({ length: 100 }, (_, index) => ({
id: `req-${String(index).padStart(3, '0')}`,
type: 'audiobook',
status: 'awaiting_search',
releaseDate: null,
audiobook: {
id: `a-${index}`,
title: `Unrelated Volume ${index}`,
author: 'Someone Else',
audibleAsin: `ASIN-${index}`,
},
}));
const secondPage = [{
id: 'req-target',
type: 'audiobook',
status: 'awaiting_search',
releaseDate: null,
audiobook: {
id: 'a-target',
title: 'Target Story',
author: 'Target Author',
audibleAsin: 'ASIN-TARGET',
},
}];
prismaMock.request.findMany
.mockResolvedValueOnce(firstPage)
.mockResolvedValueOnce(secondPage);
const { processMonitorRssFeeds } = await import('@/lib/processors/monitor-rss-feeds.processor');
const processing = processMonitorRssFeeds({ jobId: 'job-paged' });
await vi.runAllTimersAsync();
const result = await processing;
expect(result.totalMissing).toBe(101);
expect(result.matched).toBe(1);
expect(prismaMock.request.findMany).toHaveBeenNthCalledWith(2, expect.objectContaining({
orderBy: { id: 'asc' },
cursor: { id: 'req-099' },
skip: 1,
take: 100,
}));
expect(jobQueueMock.addSearchJob).toHaveBeenCalledWith(
'req-target',
expect.objectContaining({ title: 'Target Story', author: 'Target Author' })
);
vi.useRealTimers();
});
it('skips RSS auto-search when matched book is unreleased and setting ON', async () => {
@@ -77,6 +77,18 @@ describe('processRetryFailedImports', () => {
const result = await processRetryFailedImports({ jobId: 'job-1' });
expect(result.success).toBe(true);
expect(prismaMock.request.findMany).toHaveBeenCalledWith(expect.objectContaining({
orderBy: [
{ lastImportAt: { sort: 'asc', nulls: 'first' } },
{ createdAt: 'asc' },
{ id: 'asc' },
],
take: 50,
}));
expect(prismaMock.request.update).toHaveBeenCalledWith({
where: { id: 'req-1' },
data: { lastImportAt: expect.any(Date) },
});
expect(jobQueueMock.addOrganizeJob).toHaveBeenCalledWith(
'req-1',
'a1',
@@ -109,6 +121,10 @@ describe('processRetryFailedImports', () => {
expect(result.skipped).toBe(1);
expect(result.triggered).toBe(0);
expect(prismaMock.request.update).toHaveBeenCalledWith({
where: { id: 'req-2' },
data: { lastImportAt: expect.any(Date) },
});
});
it('falls back to configured download dir when qBittorrent lookup fails', async () => {
@@ -53,6 +53,23 @@ describe('processRetryMissingTorrents', () => {
const result = await processRetryMissingTorrents({ jobId: 'job-1' });
expect(result.success).toBe(true);
expect(prismaMock.request.findMany).toHaveBeenCalledWith({
where: {
deletedAt: null,
OR: [
{ status: 'awaiting_search' },
{ status: 'awaiting_release', releaseDate: null },
{ status: 'awaiting_release', releaseDate: { lte: expect.any(Date) } },
],
},
include: { audiobook: true },
orderBy: [
{ lastSearchAt: { sort: 'asc', nulls: 'first' } },
{ createdAt: 'asc' },
{ id: 'asc' },
],
take: 50,
});
expect(jobQueueMock.addSearchJob).toHaveBeenCalledWith(
'req-1',
expect.objectContaining({ id: 'a1', title: 'Book', author: 'Author' })
@@ -155,5 +172,11 @@ describe('processRetryMissingTorrents', () => {
expect(prismaMock.request.update).not.toHaveBeenCalled();
expect(jobQueueMock.addSearchJob).toHaveBeenCalled();
expect(result.triggered).toBe(1);
expect(prismaMock.request.findMany).toHaveBeenCalledWith(expect.objectContaining({
where: {
deletedAt: null,
status: { in: ['awaiting_search', 'awaiting_release'] },
},
}));
});
});
@@ -88,6 +88,17 @@ describe('processSearchIndexers', () => {
guid: 'guid-1',
format: 'M4B',
},
{
indexer: 'Fallback Indexer',
indexerId: 2,
title: 'Book - Author - Alternate',
size: 50 * 1024 * 1024,
seeders: 1,
publishDate: new Date(),
downloadUrl: 'https://prowlarr/2/download/alternate',
guid: 'guid-2',
format: 'M4B',
},
]);
prismaMock.request.update.mockResolvedValue({});
@@ -103,8 +114,55 @@ describe('processSearchIndexers', () => {
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
'req-2',
{ id: 'a2', title: 'Book', author: 'Author' },
expect.objectContaining({ title: 'Book - Author' })
expect.objectContaining({ title: 'Book - Author' }),
[expect.objectContaining({ title: 'Book - Author - Alternate' })]
);
expect(prowlarrMock.searchWithVariations).toHaveBeenCalledWith('Book', 'Author', {
categories: [3030],
indexerIds: [1],
minSeeders: 1,
});
});
it('ranks all candidates before limiting automatic search to 100 results', async () => {
configMock.get.mockImplementation(async (key: string) => {
if (key === 'prowlarr_indexers') {
return JSON.stringify([{ id: 1, name: 'Indexer', protocol: 'torrent', priority: 10, categories: [3030] }]);
}
if (key === 'indexer_flag_config') return JSON.stringify([]);
return null;
});
const candidates = Array.from({ length: 101 }, (_, index) => ({
indexer: 'Indexer',
indexerId: 1,
title: 'Book - Author',
size: 50 * 1024 * 1024,
seeders: index === 100 ? 500 : 1,
publishDate: new Date(),
downloadUrl: `magnet:?xt=urn:btih:${index}`,
guid: `guid-${index}`,
format: 'M4B',
}));
prowlarrMock.searchWithVariations.mockResolvedValue(candidates);
prismaMock.request.update.mockResolvedValue({});
const { processSearchIndexers } = await import('@/lib/processors/search-indexers.processor');
const result = await processSearchIndexers({
requestId: 'req-rank-before-limit',
audiobook: { id: 'a-rank-before-limit', title: 'Book', author: 'Author' },
jobId: 'job-rank-before-limit',
});
expect(result.success).toBe(true);
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
'req-rank-before-limit',
expect.anything(),
expect.objectContaining({ guid: 'guid-100' }),
expect.any(Array)
);
const fallbackResults = jobQueueMock.addDownloadJob.mock.calls[0][3];
expect(fallbackResults).toHaveLength(99);
});
it('fails when no indexers are configured', async () => {
@@ -179,7 +237,8 @@ describe('processSearchIndexers', () => {
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
'req-filter-name',
expect.objectContaining({ id: 'a-filter' }),
expect.objectContaining({ title: 'Good Release - Author' })
expect.objectContaining({ title: 'Good Release - Author' }),
[]
);
});
@@ -235,7 +294,8 @@ describe('processSearchIndexers', () => {
expect(jobQueueMock.addDownloadJob).toHaveBeenCalledWith(
'req-filter-hash',
expect.anything(),
expect.objectContaining({ title: 'Good Release - Author' })
expect.objectContaining({ title: 'Good Release - Author' }),
[]
);
});
+11 -2
View File
@@ -202,11 +202,20 @@ describe('JobQueueService', () => {
const { JobQueueService } = await import('@/lib/services/job-queue.service');
const service = new JobQueueService();
await service.addDownloadJob('req-1', { id: 'ab-1', title: 'Title', author: 'Author' }, { hash: 'hash' } as any);
await service.addDownloadJob(
'req-1',
{ id: 'ab-1', title: 'Title', author: 'Author' },
{ hash: 'hash' } as any,
[{ hash: 'fallback-hash' } as any]
);
expect(queueMock.add).toHaveBeenCalledWith(
'download_torrent',
expect.objectContaining({ requestId: 'req-1', jobId: 'job-2' }),
expect.objectContaining({
requestId: 'req-1',
jobId: 'job-2',
alternateTorrents: [{ hash: 'fallback-hash' }],
}),
expect.objectContaining({ priority: 9 })
);
});
@@ -38,13 +38,6 @@ vi.mock('@/lib/services/audiobookshelf/api', () => ({
deleteABSItem: vi.fn(),
}));
vi.mock('@/lib/utils/file-organizer', () => ({
buildAudiobookPath: vi.fn((mediaDir: string, template: string, data: any) => {
// Simple mock implementation that mimics the real behavior for tests
return path.join(mediaDir, data.author, `${data.title} ${data.asin}`);
}),
}));
describe('deleteRequest', () => {
beforeEach(() => {
vi.clearAllMocks();
@@ -74,6 +67,7 @@ describe('deleteRequest', () => {
audibleAsin: 'ASIN1',
plexGuid: 'plex-1',
absItemId: null,
filePath: path.join('/media', 'Author', 'Book ASIN1'),
},
downloadHistory: [
{
@@ -154,6 +148,7 @@ describe('deleteRequest', () => {
audibleAsin: null,
plexGuid: 'plex-2',
absItemId: null,
filePath: path.join('/media', 'Author', 'Book Two'),
},
downloadHistory: [
{
@@ -219,6 +214,7 @@ describe('deleteRequest', () => {
audibleAsin: 'ASIN3',
plexGuid: 'plex-3',
absItemId: null,
filePath: path.join('/media', 'Author Name', 'Book Three ASIN3'),
},
downloadHistory: [
{
@@ -291,6 +287,7 @@ describe('deleteRequest', () => {
audibleAsin: null,
plexGuid: 'plex-4',
absItemId: null,
filePath: path.join('/media', 'Author', 'Book Four'),
},
downloadHistory: [
{
@@ -349,6 +346,7 @@ describe('deleteRequest', () => {
audibleAsin: null,
plexGuid: null,
absItemId: 'abs-5',
filePath: path.join('/media', 'Author', 'Book Five'),
},
downloadHistory: [
{
@@ -401,6 +399,7 @@ describe('deleteRequest', () => {
audibleAsin: 'ASIN6',
plexGuid: null,
absItemId: 'abs-item-123',
filePath: path.join('/media', 'Author Six', 'Book Six ASIN6'),
},
downloadHistory: [],
});
@@ -447,6 +446,7 @@ describe('deleteRequest', () => {
audibleAsin: null,
plexGuid: null,
absItemId: 'abs-item-456',
filePath: path.join('/media', 'Author Seven', 'Book Seven'),
},
downloadHistory: [],
});
@@ -0,0 +1,130 @@
/**
* Component: Request Delete Stored Path Regression Tests
* Documentation: documentation/admin-features/request-deletion.md
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import * as fs from 'fs/promises';
import * as os from 'os';
import * as path from 'path';
import { createPrismaMock } from '../helpers/prisma';
const prismaMock = createPrismaMock();
const configServiceMock = {
get: vi.fn(),
getBackendMode: vi.fn(),
};
vi.mock('@/lib/db', () => ({
prisma: prismaMock,
}));
vi.mock('@/lib/services/config.service', () => ({
getConfigService: () => configServiceMock,
}));
vi.mock('@/lib/utils/logger', () => ({
RMABLogger: {
create: () => ({
info: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
}),
},
}));
describe('deleteRequest stored media path', () => {
let tempRoot: string;
beforeEach(async () => {
vi.clearAllMocks();
tempRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'rmab-request-delete-'));
configServiceMock.get.mockRejectedValue(
new Error('Media deletion must not read current path configuration')
);
configServiceMock.getBackendMode.mockResolvedValue('plex');
prismaMock.request.findMany.mockResolvedValue([]);
prismaMock.request.updateMany.mockResolvedValue({ count: 0 });
prismaMock.request.update.mockResolvedValue({});
prismaMock.audiobook.update.mockResolvedValue({});
prismaMock.plexLibrary.findMany.mockResolvedValue([]);
});
afterEach(async () => {
await fs.rm(tempRoot, { recursive: true, force: true });
});
it('deletes the stored directory and preserves the path rendered by the current template', async () => {
const storedPath = path.join(tempRoot, 'Arthur Conan Doyle', 'The Valley of Fear');
const currentTemplatePath = path.join(
tempRoot,
'Arthur Conan Doyle',
'1914 - The Valley of Fear'
);
const unrelatedFile = path.join(currentTemplatePath, 'my own rip.m4b');
await fs.mkdir(storedPath, { recursive: true });
await fs.mkdir(currentTemplatePath, { recursive: true });
await fs.writeFile(path.join(storedPath, 'book.m4b'), 'organized by RMAB');
await fs.writeFile(unrelatedFile, 'user-owned file');
prismaMock.request.findFirst.mockResolvedValue({
id: 'req-stored-path',
type: 'audiobook',
audiobook: {
id: 'ab-stored-path',
title: 'The Valley of Fear',
author: 'Arthur Conan Doyle',
narrator: null,
audibleAsin: null,
plexGuid: null,
absItemId: null,
filePath: storedPath,
fileFormat: 'm4b',
},
downloadHistory: [],
});
const { deleteRequest } = await import('@/lib/services/request-delete.service');
const result = await deleteRequest('req-stored-path', 'admin-1');
expect(result.success).toBe(true);
expect(result.filesDeleted).toBe(true);
await expect(fs.access(storedPath)).rejects.toThrow();
await expect(fs.readFile(unrelatedFile, 'utf8')).resolves.toBe('user-owned file');
expect(configServiceMock.get).not.toHaveBeenCalled();
});
it('skips file deletion when a legacy row has no stored path', async () => {
const unrelatedPath = path.join(tempRoot, 'Arthur Conan Doyle', 'The Valley of Fear');
const unrelatedFile = path.join(unrelatedPath, 'my own rip.m4b');
await fs.mkdir(unrelatedPath, { recursive: true });
await fs.writeFile(unrelatedFile, 'user-owned file');
prismaMock.request.findFirst.mockResolvedValue({
id: 'req-no-stored-path',
type: 'audiobook',
audiobook: {
id: 'ab-no-stored-path',
title: 'The Valley of Fear',
author: 'Arthur Conan Doyle',
narrator: null,
audibleAsin: null,
plexGuid: null,
absItemId: null,
filePath: null,
fileFormat: null,
},
downloadHistory: [],
});
const { deleteRequest } = await import('@/lib/services/request-delete.service');
const result = await deleteRequest('req-no-stored-path', 'admin-1');
expect(result.success).toBe(true);
expect(result.filesDeleted).toBe(false);
await expect(fs.readFile(unrelatedFile, 'utf8')).resolves.toBe('user-owned file');
expect(configServiceMock.get).not.toHaveBeenCalled();
});
});