Add find_missing_ebooks scheduled job

Introduce a safety-net scheduled job that scans completed audiobooks and auto-triggers ebook fetches for missing companions. Changes include:

- New Prisma migration + schema field: requests.ebook_auto_retry_count (nullable) to track lifetime auto-retries.
- New processor: src/lib/processors/find-missing-ebooks.processor.ts implementing the scan (limit 50), gating by ebook_auto_grab_enabled and source flags, creating ebook child requests or retrying failed ones up to a cap of 5, using transactions for race-safety and rolling back the counter if enqueue fails.
- Job queue integration: add job type, payload, processor registration, and addFindMissingEbooksJob helper.
- Scheduler integration: register the scheduled job (daily midnight) and trigger path.
- Documentation updates: backend scheduler and ebook-sidecar docs describing behavior and limits.
- Tests: add comprehensive unit tests for the processor and update scheduler tests and job-queue test helper.

This implements automated recovery for missing ebook companions while keeping the retry counter processor-private and ensuring safe concurrency handling.
This commit is contained in:
kikootwo
2026-05-17 18:22:55 -04:00
parent 6ec53ff7e3
commit 06195e6570
10 changed files with 831 additions and 4 deletions
+3 -2
View File
@@ -20,8 +20,9 @@ Manages recurring/scheduled jobs providing automated tasks (Plex scans, Audible
3. **audible_refresh** - Default: daily midnight, fetches 200 popular + 200 new releases, stores with rankings, disabled by default 3. **audible_refresh** - Default: daily midnight, fetches 200 popular + 200 new releases, stores with rankings, disabled by default
4. **retry_missing_torrents** - Default: daily midnight, processes union of `awaiting_search` `awaiting_release` (limit 50), handles both audiobook and ebook requests. Bidirectional transitions: `awaiting_search``awaiting_release` when release date is future + `indexer.skip_unreleased` ON; `awaiting_release``awaiting_search` + run search when release date has passed or setting OFF. Sole owner of these transitions. Enabled by default. 4. **retry_missing_torrents** - Default: daily midnight, processes union of `awaiting_search` `awaiting_release` (limit 50), handles both audiobook and ebook requests. Bidirectional transitions: `awaiting_search``awaiting_release` when release date is future + `indexer.skip_unreleased` ON; `awaiting_release``awaiting_search` + run search when release date has passed or setting OFF. Sole owner of these transitions. Enabled by default.
5. **retry_failed_imports** - Default: every 6 hours, re-attempts 'awaiting_import' status (limit 50), enabled by default 5. **retry_failed_imports** - Default: every 6 hours, re-attempts 'awaiting_import' status (limit 50), enabled by default
6. **cleanup_seeded_torrents** - Default: every 30 mins, deletes torrents after seeding requirements met, respects `seeding_time_minutes` config (0 = never), enabled by default 6. **find_missing_ebooks** - Default: daily midnight, scans `downloaded` `available` audiobook requests (limit 50) for missing ebook companions and triggers the existing ebook fetch flow (`addSearchEbookJob`). Gated by `ebook_auto_grab_enabled` AND at least one ebook source enabled (`ebook_annas_archive_enabled` or `ebook_indexer_search_enabled`; legacy `ebook_sidecar_enabled` accepted as Anna's fallback). Skips ebook children in-flight (`pending`, `awaiting_approval`, `searching`, `downloading`, `processing`, `awaiting_search`, `awaiting_release`) or `cancelled`. Retries `failed`/`warn` children up to **5 lifetime auto-retries** per audiobook, tracked in `Request.ebookAutoRetryCount` (nullable; processor-private — manual "Fetch Ebook" never reads/writes it). Per-candidate writes are wrapped in `prisma.$transaction` for race-safety with concurrent auto-grab; counter rolls back if `addSearchEbookJob` throws. Enabled by default. Returns `{ scanned, gapsFound, triggered, created, retried, skippedInFlight, skippedCancelled, skippedCapHit }`.
7. **monitor_rss_feeds** - Default: every 15 mins, checks RSS feeds from enabled indexers, matches against `awaiting_search` requests (audiobook and ebook, limit 100). Query is unchanged — release-date gate is applied AFTER a match is found: if matched book is unreleased + `indexer.skip_unreleased` ON, the match is skipped and request status is NOT mutated (retry job owns transitions). Enabled by default. 7. **cleanup_seeded_torrents** - Default: every 30 mins, deletes torrents after seeding requirements met, respects `seeding_time_minutes` config (0 = never), enabled by default
8. **monitor_rss_feeds** - Default: every 15 mins, checks RSS feeds from enabled indexers, matches against `awaiting_search` requests (audiobook and ebook, limit 100). Query is unchanged — release-date gate is applied AFTER a match is found: if matched book is unreleased + `indexer.skip_unreleased` ON, the match is skipped and request status is NOT mutated (retry job owns transitions). Enabled by default.
## Architecture: Bull + Cron ## Architecture: Bull + Cron
@@ -72,6 +72,10 @@ Ebooks are first-class citizens in RMAB, with their own request type, tracking,
- *Auto-grab is automatically disabled if no ebook sources are enabled. Manual fetch via admin buttons still works.* - *Auto-grab is automatically disabled if no ebook sources are enabled. Manual fetch via admin buttons still works.*
- *Kindle fix toggle only visible when preferred format is EPUB.* - *Kindle fix toggle only visible when preferred format is EPUB.*
### Safety-Net: Find Missing Ebooks Job
A scheduled `find_missing_ebooks` job (daily midnight, enabled by default) backstops the auto-grab path for cases where it silently misses books (race conditions, transient indexer failures, requests created before sources were configured, books from Goodreads/Hardcover sync). Per run it scans up to 50 audiobook requests in `downloaded`/`available` status and triggers the existing ebook fetch flow for any audiobook missing a successful ebook companion. **Lifetime auto-retry cap: 5 per audiobook** — after 5 failed auto-attempts the job stops retrying that audiobook (admin Manual "Fetch Ebook" remains available). Counter is tracked in `Request.ebookAutoRetryCount` and is **processor-private**: manual Fetch Ebook routes never read, write, or reset it. Gated by `ebook_auto_grab_enabled` AND at least one source enabled; logs no-op runs honestly. See `documentation/backend/services/scheduler.md` for full details.
### Kindle EPUB Fix ### Kindle EPUB Fix
**Purpose:** Apply compatibility fixes to EPUB files before organizing, ensuring successful Kindle import. **Purpose:** Apply compatibility fixes to EPUB files before organizing, ensuring successful Kindle import.
@@ -0,0 +1,5 @@
-- Add lifetime auto-retry counter for the find_missing_ebooks scheduled job.
-- Nullable: NULL distinguishes "never touched by this job" from 0.
-- Only the find-missing-ebooks processor reads/writes/increments this column.
-- Manual Fetch Ebook routes do not touch it (counter is sacred per engineering brief).
ALTER TABLE "requests" ADD COLUMN "ebook_auto_retry_count" INTEGER;
+1
View File
@@ -234,6 +234,7 @@ model Request {
downloadAttempts Int @default(0) @map("download_attempts") downloadAttempts Int @default(0) @map("download_attempts")
importAttempts Int @default(0) @map("import_attempts") importAttempts Int @default(0) @map("import_attempts")
maxImportRetries Int @default(5) @map("max_import_retries") maxImportRetries Int @default(5) @map("max_import_retries")
ebookAutoRetryCount Int? @map("ebook_auto_retry_count")
lastSearchAt DateTime? @map("last_search_at") lastSearchAt DateTime? @map("last_search_at")
customSearchTerms String? @map("custom_search_terms") @db.Text customSearchTerms String? @map("custom_search_terms") @db.Text
lastImportAt DateTime? @map("last_import_at") lastImportAt DateTime? @map("last_import_at")
@@ -0,0 +1,298 @@
/**
* Component: Find Missing Ebooks Processor
* Documentation: documentation/backend/services/scheduler.md
*
* Safety-net scheduled job for issue #191. Scans completed audiobook requests
* (downloaded | available) and triggers the existing ebook fetch flow for any
* audiobook whose ebook companion is missing, failed, or warned out.
*
* Gated by ebook_auto_grab_enabled AND at least one ebook source enabled.
* Per-run scan cap = 50. Per-audiobook lifetime auto-retry cap = 5
* (tracked in Request.ebookAutoRetryCount; counter is processor-private —
* manual Fetch Ebook never touches it).
*/
import { prisma } from '../db';
import { RMABLogger } from '../utils/logger';
import { getJobQueueService } from '../services/job-queue.service';
import { getConfigService } from '../services/config.service';
export interface FindMissingEbooksPayload {
jobId?: string;
scheduledJobId?: string;
}
interface CandidateRow {
parent_request_id: string;
user_id: string;
audiobook_id: string;
custom_search_terms: string | null;
audiobook_title: string;
audiobook_author: string;
audible_asin: string | null;
ebook_request_id: string | null;
ebook_status: string | null;
ebook_auto_retry_count: number | null;
}
// Statuses indicating an in-flight ebook request that must not be duplicated
// or re-triggered. `awaiting_release` is included per engineering brief's
// "include awaiting_release in the in-flight skip set" directive.
const IN_FLIGHT_STATUSES = new Set([
'pending',
'awaiting_approval',
'searching',
'downloading',
'processing',
'awaiting_search',
'awaiting_release',
]);
const AUTO_RETRY_CAP = 5;
const PER_RUN_LIMIT = 50;
export async function processFindMissingEbooks(payload: FindMissingEbooksPayload): Promise<any> {
const { jobId } = payload;
const logger = RMABLogger.forJob(jobId, 'FindMissingEbooks');
logger.info('Starting find_missing_ebooks pass');
const zeroResult = (message: string, action: 'skipped-auto-grab-off' | 'skipped-no-source') => {
logger.info(message, { action });
return {
success: true,
message,
scanned: 0,
gapsFound: 0,
triggered: 0,
created: 0,
retried: 0,
skippedInFlight: 0,
skippedCancelled: 0,
skippedCapHit: 0,
};
};
try {
const configService = getConfigService();
// Gate #1 — auto-grab feature toggle
// Default ON when key is absent/null (matches organize-files.processor.ts).
const autoGrab = await configService.get('ebook_auto_grab_enabled');
if (autoGrab === 'false') {
return zeroResult('Auto-grab disabled, skipping', 'skipped-auto-grab-off');
}
// Gate #2 — at least one ebook source enabled
// Includes legacy back-compat shim: ebook_sidecar_enabled === 'true' counts
// as Anna's Archive ON if the new key is absent (mirrors manual fetch route).
const [annasArchive, indexerSearch, legacy] = await Promise.all([
configService.get('ebook_annas_archive_enabled'),
configService.get('ebook_indexer_search_enabled'),
configService.get('ebook_sidecar_enabled'),
]);
const annasOn = annasArchive === 'true' || (annasArchive == null && legacy === 'true');
const indexerOn = indexerSearch === 'true';
if (!annasOn && !indexerOn) {
return zeroResult('No ebook sources enabled, skipping', 'skipped-no-source');
}
// Anti-join: most-recent non-deleted ebook child per in-scope audiobook.
// Broad form — branch fully in JS so per-skip counters and log lines are
// observable. LIMIT is the per-run scan cap.
const candidates = await prisma.$queryRaw<CandidateRow[]>`
SELECT
p.id AS parent_request_id,
p.user_id AS user_id,
p.audiobook_id AS audiobook_id,
p.custom_search_terms AS custom_search_terms,
a.title AS audiobook_title,
a.author AS audiobook_author,
a.audible_asin AS audible_asin,
e.id AS ebook_request_id,
e.status AS ebook_status,
e.ebook_auto_retry_count AS ebook_auto_retry_count
FROM requests p
JOIN audiobooks a ON a.id = p.audiobook_id
LEFT JOIN LATERAL (
SELECT id, status, ebook_auto_retry_count
FROM requests
WHERE parent_request_id = p.id
AND type = 'ebook'
AND deleted_at IS NULL
ORDER BY created_at DESC
LIMIT 1
) e ON TRUE
WHERE p.status IN ('downloaded', 'available')
AND (p.type IS NULL OR p.type <> 'ebook')
AND p.deleted_at IS NULL
ORDER BY p.created_at DESC
LIMIT ${PER_RUN_LIMIT}
`;
const scanned = candidates.length;
logger.info(`Scanned ${scanned} in-scope audiobook request(s)`);
if (scanned === 0) {
return {
success: true,
message: 'No in-scope audiobook requests',
scanned: 0,
gapsFound: 0,
triggered: 0,
created: 0,
retried: 0,
skippedInFlight: 0,
skippedCancelled: 0,
skippedCapHit: 0,
};
}
const jobQueue = getJobQueueService();
let gapsFound = 0;
let created = 0;
let retried = 0;
let skippedInFlight = 0;
let skippedCancelled = 0;
let skippedCapHit = 0;
for (const row of candidates) {
let action:
| 'created'
| 'retried'
| 'skipped-has-companion'
| 'skipped-in-flight'
| 'skipped-cancelled'
| 'skipped-cap'
| 'skipped-unknown'
| null = null;
let ebookRequestId: string | null = row.ebook_request_id;
try {
await prisma.$transaction(async (tx) => {
if (!row.ebook_request_id) {
// No live ebook child — create one and seed counter at 1.
const createdRow = await tx.request.create({
data: {
userId: row.user_id,
audiobookId: row.audiobook_id,
type: 'ebook',
parentRequestId: row.parent_request_id,
status: 'pending',
progress: 0,
customSearchTerms: row.custom_search_terms,
ebookAutoRetryCount: 1,
},
});
ebookRequestId = createdRow.id;
action = 'created';
return;
}
const status = row.ebook_status;
if (status === 'downloaded') {
action = 'skipped-has-companion';
return;
}
if (status && IN_FLIGHT_STATUSES.has(status)) {
action = 'skipped-in-flight';
return;
}
if (status === 'cancelled') {
action = 'skipped-cancelled';
return;
}
if (status === 'failed' || status === 'warn') {
const current = row.ebook_auto_retry_count ?? 0;
if (current >= AUTO_RETRY_CAP) {
action = 'skipped-cap';
return;
}
await tx.request.update({
where: { id: row.ebook_request_id! },
data: {
status: 'pending',
progress: 0,
errorMessage: null,
ebookAutoRetryCount: current + 1,
},
});
action = 'retried';
return;
}
// Defensive — unrecognized status (e.g. denied, awaiting_import on an
// ebook child that crossed wires). Leave it alone; surface via log.
action = 'skipped-unknown';
});
if (action === 'created' || action === 'retried') {
gapsFound++;
try {
await jobQueue.addSearchEbookJob(ebookRequestId!, {
id: row.audiobook_id,
title: row.audiobook_title,
author: row.audiobook_author,
asin: row.audible_asin || undefined,
});
if (action === 'created') created++;
else retried++;
} catch (enqueueErr) {
// Roll counter back on enqueue failure so the cap reflects only
// successful auto-retries. Per engineering brief: "increment only
// when queue add succeeds." Failure to decrement is logged but
// swallowed — primary error is the one that matters.
await prisma.request.update({
where: { id: ebookRequestId! },
data: { ebookAutoRetryCount: { decrement: 1 } },
}).catch((rollbackErr) => {
logger.error(`Failed to roll back counter for ebook ${ebookRequestId}: ${rollbackErr instanceof Error ? rollbackErr.message : 'Unknown error'}`);
});
throw enqueueErr;
}
} else if (action === 'skipped-in-flight') skippedInFlight++;
else if (action === 'skipped-cancelled') skippedCancelled++;
else if (action === 'skipped-cap') skippedCapHit++;
logger.info('find_missing_ebooks iteration', {
audiobookId: row.audiobook_id,
parentRequestId: row.parent_request_id,
ebookRequestId,
action,
});
} catch (err) {
logger.error(`Failed candidate ${row.parent_request_id}: ${err instanceof Error ? err.message : 'Unknown error'}`);
}
// Spread DB operations over time to avoid connection pool exhaustion.
await new Promise((resolve) => setTimeout(resolve, 100));
}
const triggered = created + retried;
logger.info('find_missing_ebooks pass complete', {
scanned,
gapsFound,
triggered,
created,
retried,
skippedInFlight,
skippedCancelled,
skippedCapHit,
});
return {
success: true,
message: 'find_missing_ebooks completed',
scanned,
gapsFound,
triggered,
created,
retried,
skippedInFlight,
skippedCancelled,
skippedCapHit,
};
} catch (error) {
logger.error(`Error: ${error instanceof Error ? error.message : 'Unknown error'}`);
throw error;
}
}
+26
View File
@@ -24,6 +24,7 @@ export type JobType =
| 'audible_refresh' | 'audible_refresh'
| 'retry_missing_torrents' | 'retry_missing_torrents'
| 'retry_failed_imports' | 'retry_failed_imports'
| 'find_missing_ebooks'
| 'cleanup_seeded_torrents' | 'cleanup_seeded_torrents'
| 'monitor_rss_feeds' | 'monitor_rss_feeds'
| 'sync_reading_shelves' | 'sync_reading_shelves'
@@ -105,6 +106,10 @@ export interface RetryFailedImportsPayload extends JobPayload {
scheduledJobId?: string; scheduledJobId?: string;
} }
export interface FindMissingEbooksPayload extends JobPayload {
scheduledJobId?: string;
}
export interface CleanupSeededTorrentsPayload extends JobPayload { export interface CleanupSeededTorrentsPayload extends JobPayload {
scheduledJobId?: string; scheduledJobId?: string;
} }
@@ -386,6 +391,12 @@ export class JobQueueService {
return await processRetryFailedImports(payloadWithJobId); return await processRetryFailedImports(payloadWithJobId);
}); });
this.queue.process('find_missing_ebooks', 1, async (job: BullJob<FindMissingEbooksPayload>) => {
const { processFindMissingEbooks } = await import('../processors/find-missing-ebooks.processor');
const payloadWithJobId = await this.ensureJobRecord(job, 'find_missing_ebooks');
return await processFindMissingEbooks(payloadWithJobId);
});
this.queue.process('cleanup_seeded_torrents', 1, async (job: BullJob<CleanupSeededTorrentsPayload>) => { this.queue.process('cleanup_seeded_torrents', 1, async (job: BullJob<CleanupSeededTorrentsPayload>) => {
const { processCleanupSeededTorrents } = await import('../processors/cleanup-seeded-torrents.processor'); const { processCleanupSeededTorrents } = await import('../processors/cleanup-seeded-torrents.processor');
const payloadWithJobId = await this.ensureJobRecord(job, 'cleanup_seeded_torrents'); const payloadWithJobId = await this.ensureJobRecord(job, 'cleanup_seeded_torrents');
@@ -756,6 +767,21 @@ export class JobQueueService {
); );
} }
/**
* Add find missing ebooks job
*/
async addFindMissingEbooksJob(scheduledJobId?: string): Promise<string> {
return await this.addJob(
'find_missing_ebooks',
{
scheduledJobId,
} as FindMissingEbooksPayload,
{
priority: 7,
}
);
}
/** /**
* Add cleanup seeded torrents job * Add cleanup seeded torrents job
*/ */
+18 -1
View File
@@ -10,7 +10,7 @@ import { RMABLogger } from '../utils/logger';
const logger = RMABLogger.create('Scheduler'); const logger = RMABLogger.create('Scheduler');
export type ScheduledJobType = 'plex_library_scan' | 'plex_recently_added_check' | 'audible_refresh' | 'retry_missing_torrents' | 'retry_failed_imports' | 'cleanup_seeded_torrents' | 'monitor_rss_feeds' | 'sync_reading_shelves' | 'check_watched_lists'; export type ScheduledJobType = 'plex_library_scan' | 'plex_recently_added_check' | 'audible_refresh' | 'retry_missing_torrents' | 'retry_failed_imports' | 'find_missing_ebooks' | 'cleanup_seeded_torrents' | 'monitor_rss_feeds' | 'sync_reading_shelves' | 'check_watched_lists';
export interface ScheduledJob { export interface ScheduledJob {
id: string; id: string;
@@ -115,6 +115,13 @@ export class SchedulerService {
enabled: true, // Enable by default enabled: true, // Enable by default
payload: {}, payload: {},
}, },
{
name: 'Find Missing Ebooks',
type: 'find_missing_ebooks' as ScheduledJobType,
schedule: '0 0 * * *', // Daily at midnight
enabled: true, // Enable by default; gated by ebook_auto_grab_enabled + source-enablement at run time
payload: {},
},
{ {
name: 'Cleanup Seeded Torrents', name: 'Cleanup Seeded Torrents',
type: 'cleanup_seeded_torrents' as ScheduledJobType, type: 'cleanup_seeded_torrents' as ScheduledJobType,
@@ -379,6 +386,9 @@ export class SchedulerService {
case 'retry_failed_imports': case 'retry_failed_imports':
bullJobId = await this.triggerRetryFailedImports(job); bullJobId = await this.triggerRetryFailedImports(job);
break; break;
case 'find_missing_ebooks':
bullJobId = await this.triggerFindMissingEbooks(job);
break;
case 'cleanup_seeded_torrents': case 'cleanup_seeded_torrents':
bullJobId = await this.triggerCleanupSeededTorrents(job); bullJobId = await this.triggerCleanupSeededTorrents(job);
break; break;
@@ -645,6 +655,13 @@ export class SchedulerService {
return await this.jobQueue.addRetryFailedImportsJob(job.id); return await this.jobQueue.addRetryFailedImportsJob(job.id);
} }
/**
* Trigger find missing ebooks safety-net pass
*/
private async triggerFindMissingEbooks(job: any): Promise<string> {
return await this.jobQueue.addFindMissingEbooksJob(job.id);
}
/** /**
* Trigger RSS feed monitoring * Trigger RSS feed monitoring
*/ */
+1
View File
@@ -18,6 +18,7 @@ export const createJobQueueMock = () => ({
addAudibleRefreshJob: vi.fn(), addAudibleRefreshJob: vi.fn(),
addRetryMissingTorrentsJob: vi.fn(), addRetryMissingTorrentsJob: vi.fn(),
addRetryFailedImportsJob: vi.fn(), addRetryFailedImportsJob: vi.fn(),
addFindMissingEbooksJob: vi.fn(),
addCleanupSeededTorrentsJob: vi.fn(), addCleanupSeededTorrentsJob: vi.fn(),
addNotificationJob: vi.fn().mockResolvedValue(undefined), addNotificationJob: vi.fn().mockResolvedValue(undefined),
}); });
@@ -0,0 +1,472 @@
/**
* Component: Find Missing Ebooks Processor Tests
* Documentation: documentation/backend/services/scheduler.md
*/
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { createPrismaMock } from '../helpers/prisma';
import { createJobQueueMock } from '../helpers/job-queue';
const prismaMock = createPrismaMock();
const jobQueueMock = createJobQueueMock();
const configMock = vi.hoisted(() => ({ get: vi.fn() }));
vi.mock('@/lib/db', () => ({
prisma: prismaMock,
}));
vi.mock('@/lib/services/job-queue.service', () => ({
getJobQueueService: () => jobQueueMock,
}));
vi.mock('@/lib/services/config.service', () => ({
getConfigService: () => configMock,
}));
type CandidateRow = {
parent_request_id: string;
user_id: string;
audiobook_id: string;
custom_search_terms: string | null;
audiobook_title: string;
audiobook_author: string;
audible_asin: string | null;
ebook_request_id: string | null;
ebook_status: string | null;
ebook_auto_retry_count: number | null;
};
const baseRow = (overrides: Partial<CandidateRow> = {}): CandidateRow => ({
parent_request_id: 'parent-1',
user_id: 'user-1',
audiobook_id: 'audio-1',
custom_search_terms: null,
audiobook_title: 'Test Book',
audiobook_author: 'Test Author',
audible_asin: 'ASIN0001',
ebook_request_id: null,
ebook_status: null,
ebook_auto_retry_count: null,
...overrides,
});
/**
* Default: all gates pass (auto-grab default ON when null; Anna's enabled).
* Tests that want a different gate state can override before calling.
*/
const installDefaultGates = () => {
configMock.get.mockImplementation(async (key: string) => {
switch (key) {
case 'ebook_auto_grab_enabled':
return null; // null/absent => ON
case 'ebook_annas_archive_enabled':
return 'true';
case 'ebook_indexer_search_enabled':
return 'false';
case 'ebook_sidecar_enabled':
return null;
default:
return null;
}
});
};
beforeEach(() => {
vi.clearAllMocks();
installDefaultGates();
// Default: $transaction runs the callback against the prismaMock surface.
prismaMock.$transaction.mockImplementation(async (fn: any) => fn(prismaMock));
// Default: each create call returns a stable ebook request id.
prismaMock.request.create.mockImplementation(async (args: any) => ({
id: 'new-ebook-1',
...args.data,
}));
prismaMock.request.update.mockResolvedValue({});
});
describe('processFindMissingEbooks — gating', () => {
it('returns zeros when auto-grab is disabled (explicit false)', async () => {
configMock.get.mockImplementation(async (key: string) =>
key === 'ebook_auto_grab_enabled' ? 'false' : null
);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-1' });
expect(result).toMatchObject({
success: true,
scanned: 0,
gapsFound: 0,
triggered: 0,
created: 0,
retried: 0,
skippedInFlight: 0,
skippedCancelled: 0,
skippedCapHit: 0,
});
expect(prismaMock.$queryRaw).not.toHaveBeenCalled();
expect(jobQueueMock.addSearchEbookJob).not.toHaveBeenCalled();
});
it('treats auto-grab unset (null) as ON and proceeds to source check', async () => {
configMock.get.mockImplementation(async (key: string) => {
switch (key) {
case 'ebook_auto_grab_enabled':
return null;
case 'ebook_annas_archive_enabled':
return 'true';
default:
return null;
}
});
prismaMock.$queryRaw.mockResolvedValue([]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-2' });
expect(result.scanned).toBe(0);
expect(prismaMock.$queryRaw).toHaveBeenCalledTimes(1);
});
it('returns zeros when both new source keys disabled AND no legacy key', async () => {
configMock.get.mockImplementation(async (key: string) => {
switch (key) {
case 'ebook_auto_grab_enabled':
return null;
case 'ebook_annas_archive_enabled':
return 'false';
case 'ebook_indexer_search_enabled':
return 'false';
case 'ebook_sidecar_enabled':
return null;
default:
return null;
}
});
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-3' });
expect(result.scanned).toBe(0);
expect(prismaMock.$queryRaw).not.toHaveBeenCalled();
});
it('legacy ebook_sidecar_enabled=true (with new keys absent) passes the source gate', async () => {
configMock.get.mockImplementation(async (key: string) => {
switch (key) {
case 'ebook_auto_grab_enabled':
return null;
case 'ebook_annas_archive_enabled':
return null;
case 'ebook_indexer_search_enabled':
return null;
case 'ebook_sidecar_enabled':
return 'true';
default:
return null;
}
});
prismaMock.$queryRaw.mockResolvedValue([]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-4' });
expect(result.scanned).toBe(0);
expect(prismaMock.$queryRaw).toHaveBeenCalledTimes(1);
});
});
describe('processFindMissingEbooks — fresh-gap creation', () => {
it('creates a new ebook request when no live ebook child exists (audiobook downloaded)', async () => {
prismaMock.$queryRaw.mockResolvedValue([
baseRow({
parent_request_id: 'parent-fresh',
user_id: 'user-x',
audiobook_id: 'audio-x',
audiobook_title: 'Fresh Book',
audiobook_author: 'Some Author',
audible_asin: 'B09ABCDEFG',
custom_search_terms: 'cst',
}),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-5' });
expect(prismaMock.request.create).toHaveBeenCalledWith({
data: {
userId: 'user-x',
audiobookId: 'audio-x',
type: 'ebook',
parentRequestId: 'parent-fresh',
status: 'pending',
progress: 0,
customSearchTerms: 'cst',
ebookAutoRetryCount: 1,
},
});
expect(jobQueueMock.addSearchEbookJob).toHaveBeenCalledWith(
'new-ebook-1',
expect.objectContaining({
id: 'audio-x',
title: 'Fresh Book',
author: 'Some Author',
asin: 'B09ABCDEFG',
})
);
expect(result).toMatchObject({
scanned: 1,
gapsFound: 1,
triggered: 1,
created: 1,
retried: 0,
skippedInFlight: 0,
skippedCancelled: 0,
skippedCapHit: 0,
});
});
it('also creates for audiobook in `available` state (both statuses in scope)', async () => {
// The query is responsible for filtering by status; here we just confirm
// that the processor doesn't add a second status guard in JS that would
// reject a row coming back from SQL.
prismaMock.$queryRaw.mockResolvedValue([
baseRow({
parent_request_id: 'parent-available',
audiobook_title: 'Available Book',
}),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-6' });
expect(result.created).toBe(1);
expect(prismaMock.request.create).toHaveBeenCalled();
});
it('omits asin when audiobook has no audibleAsin', async () => {
prismaMock.$queryRaw.mockResolvedValue([baseRow({ audible_asin: null })]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
await processFindMissingEbooks({ jobId: 'job-6b' });
expect(jobQueueMock.addSearchEbookJob).toHaveBeenCalledWith(
'new-ebook-1',
expect.objectContaining({ asin: undefined })
);
});
});
describe('processFindMissingEbooks — branch skips', () => {
it('skips when most-recent ebook child is already downloaded (defensive)', async () => {
prismaMock.$queryRaw.mockResolvedValue([
baseRow({
ebook_request_id: 'ebook-1',
ebook_status: 'downloaded',
ebook_auto_retry_count: 0,
}),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-7' });
expect(prismaMock.request.create).not.toHaveBeenCalled();
expect(prismaMock.request.update).not.toHaveBeenCalled();
expect(jobQueueMock.addSearchEbookJob).not.toHaveBeenCalled();
// skipped-has-companion is intentionally not surfaced as its own counter —
// admin can derive from scanned - gapsFound - skippedInFlight - skippedCancelled - skippedCapHit.
expect(result).toMatchObject({
scanned: 1,
gapsFound: 0,
triggered: 0,
skippedInFlight: 0,
skippedCancelled: 0,
skippedCapHit: 0,
});
});
it.each([
'pending',
'awaiting_approval',
'searching',
'downloading',
'processing',
'awaiting_search',
'awaiting_release',
])('skips when most-recent ebook child status is in-flight: %s', async (status) => {
prismaMock.$queryRaw.mockResolvedValue([
baseRow({ ebook_request_id: 'ebook-1', ebook_status: status }),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: `job-inflight-${status}` });
expect(prismaMock.request.create).not.toHaveBeenCalled();
expect(prismaMock.request.update).not.toHaveBeenCalled();
expect(jobQueueMock.addSearchEbookJob).not.toHaveBeenCalled();
expect(result.skippedInFlight).toBe(1);
expect(result.gapsFound).toBe(0);
});
it('skips when most-recent ebook child status is cancelled', async () => {
prismaMock.$queryRaw.mockResolvedValue([
baseRow({ ebook_request_id: 'ebook-1', ebook_status: 'cancelled' }),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-cancelled' });
expect(jobQueueMock.addSearchEbookJob).not.toHaveBeenCalled();
expect(result.skippedCancelled).toBe(1);
});
});
describe('processFindMissingEbooks — retry path', () => {
it('retries a failed ebook child with counter < cap, increments counter', async () => {
prismaMock.$queryRaw.mockResolvedValue([
baseRow({
ebook_request_id: 'ebook-fail-1',
ebook_status: 'failed',
ebook_auto_retry_count: 3,
}),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-retry' });
expect(prismaMock.request.update).toHaveBeenCalledWith({
where: { id: 'ebook-fail-1' },
data: {
status: 'pending',
progress: 0,
errorMessage: null,
ebookAutoRetryCount: 4,
},
});
expect(jobQueueMock.addSearchEbookJob).toHaveBeenCalledWith(
'ebook-fail-1',
expect.objectContaining({ id: 'audio-1' })
);
expect(result).toMatchObject({
retried: 1,
created: 0,
gapsFound: 1,
triggered: 1,
skippedCapHit: 0,
});
});
it('skips a warn ebook child whose counter is at the cap (5)', async () => {
prismaMock.$queryRaw.mockResolvedValue([
baseRow({
ebook_request_id: 'ebook-cap',
ebook_status: 'warn',
ebook_auto_retry_count: 5,
}),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-cap' });
expect(prismaMock.request.update).not.toHaveBeenCalled();
expect(jobQueueMock.addSearchEbookJob).not.toHaveBeenCalled();
expect(result.skippedCapHit).toBe(1);
expect(result.retried).toBe(0);
});
it('retries a failed ebook child with null counter, sets counter to 1', async () => {
prismaMock.$queryRaw.mockResolvedValue([
baseRow({
ebook_request_id: 'ebook-null-counter',
ebook_status: 'failed',
ebook_auto_retry_count: null,
}),
]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-null' });
expect(prismaMock.request.update).toHaveBeenCalledWith({
where: { id: 'ebook-null-counter' },
data: {
status: 'pending',
progress: 0,
errorMessage: null,
ebookAutoRetryCount: 1,
},
});
expect(result.retried).toBe(1);
});
});
describe('processFindMissingEbooks — error isolation', () => {
it('rolls back the counter when addSearchEbookJob throws, then continues with next candidate', async () => {
// Two candidates: first one's enqueue will throw, second should still process.
prismaMock.$queryRaw.mockResolvedValue([
baseRow({
parent_request_id: 'parent-throw',
audiobook_id: 'audio-throw',
ebook_request_id: 'ebook-throw',
ebook_status: 'failed',
ebook_auto_retry_count: 2,
}),
baseRow({
parent_request_id: 'parent-ok',
audiobook_id: 'audio-ok',
ebook_request_id: 'ebook-ok',
ebook_status: 'failed',
ebook_auto_retry_count: 0,
}),
]);
jobQueueMock.addSearchEbookJob
.mockRejectedValueOnce(new Error('queue blew up'))
.mockResolvedValueOnce('bull-job-id');
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-throw' });
// Counter rolled back on the throwing candidate:
expect(prismaMock.request.update).toHaveBeenCalledWith({
where: { id: 'ebook-throw' },
data: { ebookAutoRetryCount: { decrement: 1 } },
});
// Second candidate still processed:
expect(prismaMock.request.update).toHaveBeenCalledWith({
where: { id: 'ebook-ok' },
data: {
status: 'pending',
progress: 0,
errorMessage: null,
ebookAutoRetryCount: 1,
},
});
// gapsFound counts both attempted gaps; only the second succeeds in being triggered.
expect(result.gapsFound).toBe(2);
expect(result.retried).toBe(1);
expect(result.created).toBe(0);
});
});
describe('processFindMissingEbooks — return shape', () => {
it('exposes all observable counters in the result', async () => {
prismaMock.$queryRaw.mockResolvedValue([]);
const { processFindMissingEbooks } = await import('@/lib/processors/find-missing-ebooks.processor');
const result = await processFindMissingEbooks({ jobId: 'job-shape' });
expect(result).toEqual(
expect.objectContaining({
success: true,
scanned: 0,
gapsFound: 0,
triggered: 0,
created: 0,
retried: 0,
skippedInFlight: 0,
skippedCancelled: 0,
skippedCapHit: 0,
})
);
});
});
+3 -1
View File
@@ -16,6 +16,7 @@ const jobQueueMock = vi.hoisted(() => ({
addAudibleRefreshJob: vi.fn(), addAudibleRefreshJob: vi.fn(),
addRetryMissingTorrentsJob: vi.fn(), addRetryMissingTorrentsJob: vi.fn(),
addRetryFailedImportsJob: vi.fn(), addRetryFailedImportsJob: vi.fn(),
addFindMissingEbooksJob: vi.fn(),
addCleanupSeededTorrentsJob: vi.fn(), addCleanupSeededTorrentsJob: vi.fn(),
addMonitorRssFeedsJob: vi.fn(), addMonitorRssFeedsJob: vi.fn(),
addSyncShelvesJob: vi.fn(), addSyncShelvesJob: vi.fn(),
@@ -80,7 +81,7 @@ describe('SchedulerService', () => {
const service = new SchedulerService(); const service = new SchedulerService();
await service.start(); await service.start();
expect(prismaMock.scheduledJob.create).toHaveBeenCalledTimes(9); expect(prismaMock.scheduledJob.create).toHaveBeenCalledTimes(10);
expect(jobQueueMock.addRepeatableJob).toHaveBeenCalledWith( expect(jobQueueMock.addRepeatableJob).toHaveBeenCalledWith(
'audible_refresh', 'audible_refresh',
{ scheduledJobId: 'job-1' }, { scheduledJobId: 'job-1' },
@@ -289,6 +290,7 @@ describe('SchedulerService', () => {
['audible_refresh', 'addAudibleRefreshJob'], ['audible_refresh', 'addAudibleRefreshJob'],
['retry_missing_torrents', 'addRetryMissingTorrentsJob'], ['retry_missing_torrents', 'addRetryMissingTorrentsJob'],
['retry_failed_imports', 'addRetryFailedImportsJob'], ['retry_failed_imports', 'addRetryFailedImportsJob'],
['find_missing_ebooks', 'addFindMissingEbooksJob'],
['cleanup_seeded_torrents', 'addCleanupSeededTorrentsJob'], ['cleanup_seeded_torrents', 'addCleanupSeededTorrentsJob'],
['monitor_rss_feeds', 'addMonitorRssFeedsJob'], ['monitor_rss_feeds', 'addMonitorRssFeedsJob'],
['sync_reading_shelves', 'addSyncShelvesJob'], ['sync_reading_shelves', 'addSyncShelvesJob'],