Compare commits

...

11 Commits

Author SHA1 Message Date
kikootwo 04dbb05a6e Bump package version to 1.0.9
Update package.json version from 1.0.8 to 1.0.9 to mark a new patch release. No other changes in this diff.
2026-02-20 10:19:50 -05:00
kikootwo cb9f1b81bc Add series browsing, search, and detail UI
Introduce full support for Audible series exploration: API routes, frontend pages, components, hooks, and integrations. Key changes:

- Prisma: add Audiobook.seriesAsin for linking audiobooks to series detail pages.
- Backend: add /api/series/search and /api/series/[asin] routes that require auth; scrape Audible series data and enrich books with library availability.
- Integrations/services: add audible-series integration and update request/HTTP services to support the workflow.
- Frontend: add /series and /series/[asin] pages, new components (SeriesCard, SeriesGrid, SeriesDetailCard, SimilarSeriesRow) and wire them to a new useSeries hook; update AudiobookDetailsModal to show/link series; add Series link to Header.
- Misc: extend audiobook types with series fields and add seriesLabels to language-config for scraping.

These changes enable users to search for series, view series metadata and books, and navigate between audiobook and series detail pages.
2026-02-20 10:19:30 -05:00
kikootwo 5d8ac2f73d Add language config and locale-aware parsing
Introduce centralized language configuration and wire locale-aware behavior across scraping and ranking. Adds src/lib/constants/language-config.ts with per-language scraping rules, stop words, and character replacements; replaces AudibleRegion.isEnglish with a language field in types and AUDIBLE_REGIONS. Update AudibleService, ebook scraper, processors, and API routes to use getLanguageForRegion so Anna's Archive searches, scraping selectors, runtime/rating parsing, and ranking use language-specific params and filters. Extend ranking algorithm to accept stopWords and characterReplacements and apply them during normalization and matching. Update UI selects to mark non-English regions and adjust tests accordingly.
2026-02-20 06:32:44 -05:00
kikootwo c146383735 Don't coerce customPath to undefined
Pass sanitizedCustomPath through directly instead of using `sanitizedCustomPath || undefined`. The previous fallback converted falsy values (e.g. empty string) to undefined; this change preserves explicit empty or falsy values so downstream logic can distinguish them from undefined.
2026-02-18 17:51:35 -05:00
kikootwo 3820b9b21d Add DB pooling, throttling and monitor backoff
Add connection pool params to DATABASE_URL and configure Prisma to use the pooled URL (connection_limit=20, pool_timeout=30) to reduce connection exhaustion. Introduce safeguards and throttling across processors: limit in-flight progress DB updates in direct-download, add short delays when processing RSS, retry-failed-imports, and retry-missing-torrents, and stagger scheduler triggers to avoid bursts. Implement adaptive monitor-download polling with stallCount/lastProgress and exponential backoff, and thread these fields through JobQueueService (including reduced worker concurrency for several queues). Batch audiobook enrichment queries to small parallel batches to limit DB load. Update tests to reflect new monitor payload parameters. Overall intent: reduce DB connection pool pressure and smooth load spikes during startup and heavy processing.
2026-02-18 02:43:00 -05:00
kikootwo 20798b3dc0 Add RDT-Client support and Prowlarr prompt
Introduce RDT-Client integration and related UI/behavior changes.

- Add RDTClientService extending QBittorrentService with RDT-specific behavior (stale-torrent deletion, postProcess cleanup, no-op categories).
- Register 'rdtclient' in supported client types, display names, and protocol mapping; create RDT client factory in DownloadClientManager.
- Add RDT-Client card to DownloadClientManagement UI and placeholder URL in DownloadClientModal.
- Update qbittorrent service: omit per-torrent savepath/sequential options (favor category/automatic management), make several methods protected, and clean up related comments.
- Make organize-files.processor treat rdtclient as a special-case for cleanup (remove local torrent entries after organize).
- Add prowlarr service singleton invalidation and call it when Prowlarr settings are updated so background jobs pick up new credentials.
- Add confirmation flow when changing Prowlarr URL/API key: new useIndexersSettings logic to detect credential changes, prompt ConfirmModal from IndexersTab, and optionally clear configured indexers on confirmed change.

These changes ensure Real-Debrid-backed qBittorrent-compatible clients are supported correctly and that switching Prowlarr credentials is handled safely.
2026-02-17 17:03:21 -05:00
kikootwo 3f8180a246 Add server readiness check & init retries
Wait for the Next.js server and DB to be healthy before initializing services in docker/unified/app-start.sh. Adds a health probe with configurable timeout and retries, backoff retries for the /api/init call, improved logging, and error handling when the server process exits.

In src/lib/services/scheduler.service.ts, make re-encryption of notification backends non-fatal by catching and logging errors, and make creation of default scheduled jobs robust by creating each job independently with per-job error handling and logging. Summary counts are logged for created/failed defaults so failures don't block the scheduler from starting.
2026-02-13 14:03:21 -05:00
kikootwo c97df7798a Merge pull request #78 from gtronset/feature/unraid-template-v2
Update Unriad Template to fix permissions and fix `WebUI` link
2026-02-12 23:11:02 -05:00
Gavin Tronset c0096cda1a Update Unriad Template to fix permisions and WebUI link 2026-02-12 16:20:59 -08:00
kikootwo 98a2cc2813 Mock getBaseUrl in Audible service tests
Add a getBaseUrl mock to audibleServiceMock in audiobooks-browse route tests that returns 'https://www.audible.com'. This ensures tests have a defined base URL for Audible service calls and prevents issues caused by an undefined method during test execution.
2026-02-12 16:09:55 -05:00
kikootwo 4df49633b4 Bump package version to 1.0.8
Update package.json version from 1.0.7 to 1.0.8 to prepare a patch release.
2026-02-12 16:00:45 -05:00
59 changed files with 2717 additions and 193 deletions
+66 -5
View File
@@ -53,14 +53,75 @@ start_server() {
start_server
SERVER_PID=$!
echo "[App] Waiting for server to be ready..."
sleep 5
# =============================================================================
# WAIT FOR SERVER READINESS
# =============================================================================
# The health endpoint (/api/health) checks both the Next.js server AND database
# connectivity. We must wait for both before initializing scheduled jobs.
HEALTH_URL="http://localhost:3030/api/health"
INIT_URL="http://localhost:3030/api/init"
READY_TIMEOUT=${APP_READY_TIMEOUT:-60}
INIT_RETRIES=${APP_INIT_RETRIES:-5}
echo "[App] Waiting for server to be ready (timeout: ${READY_TIMEOUT}s)..."
READY=false
for i in $(seq 1 "$READY_TIMEOUT"); do
# Check if the server process is still alive
if ! kill -0 "$SERVER_PID" 2>/dev/null; then
echo "[App] ERROR: Server process (PID $SERVER_PID) exited unexpectedly"
exit 1
fi
if curl -sf "$HEALTH_URL" > /dev/null 2>&1; then
READY=true
echo "[App] Server is healthy (took ${i}s)"
break
fi
# Log progress every 10 seconds
if [ $((i % 10)) -eq 0 ]; then
echo "[App] Still waiting for server... (${i}/${READY_TIMEOUT}s)"
fi
sleep 1
done
if [ "$READY" = "false" ]; then
echo "[App] ERROR: Server did not become healthy within ${READY_TIMEOUT}s"
echo "[App] The scheduler will not be initialized - scheduled jobs may be missing"
echo "[App] Check server logs above for errors (database connection, port conflict, etc.)"
else
# =========================================================================
# INITIALIZE APPLICATION SERVICES
# =========================================================================
# Creates default scheduled jobs, runs credential migration, etc.
# Retry with backoff to handle transient failures during startup.
# Initialize application services (creates default scheduled jobs)
echo "[App] Initializing application services..."
curl -sf http://localhost:3030/api/init || echo "[App] Warning: Failed to initialize services (may already be initialized)"
echo "[App] Server ready with PID $SERVER_PID"
INIT_SUCCESS=false
for attempt in $(seq 1 "$INIT_RETRIES"); do
HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" "$INIT_URL" 2>/dev/null) || HTTP_CODE="000"
if [ "$HTTP_CODE" = "200" ]; then
INIT_SUCCESS=true
echo "[App] Services initialized successfully"
break
fi
echo "[App] Init attempt $attempt/$INIT_RETRIES failed (HTTP $HTTP_CODE), retrying in ${attempt}s..."
sleep "$attempt"
done
if [ "$INIT_SUCCESS" = "false" ]; then
echo "[App] ERROR: Failed to initialize services after $INIT_RETRIES attempts"
echo "[App] Scheduled jobs may be missing - check application logs for details"
fi
fi
echo "[App] Server running with PID $SERVER_PID"
# Verify the process is running with correct UID:GID (for debugging)
if [ -f "/proc/$SERVER_PID/status" ]; then
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "readmeabook",
"version": "1.0.7",
"version": "1.0.9",
"private": true,
"scripts": {
"dev": "next dev",
+1
View File
@@ -176,6 +176,7 @@ model Audiobook {
year Int? // Release year extracted from releaseDate
series String? // Book series name (e.g., "The Mistborn Saga")
seriesPart String? @map("series_part") // Series position (e.g., "1", "1.5", "Book 1")
seriesAsin String? @map("series_asin") // Audible series ASIN for linking to series detail page
// Request tracking
status String @default("requested") // requested, downloading, processing, completed, failed
+1
View File
@@ -295,6 +295,7 @@ export default function AdminSettings() {
{activeTab === 'prowlarr' && (
<IndexersTab
settings={settings}
originalSettings={originalSettings}
indexers={configuredIndexers}
flagConfigs={flagConfigs}
onChange={setSettings}
@@ -7,6 +7,7 @@
import React, { useEffect } from 'react';
import { Button } from '@/components/ui/Button';
import { ConfirmModal } from '@/components/ui/ConfirmModal';
import { Input } from '@/components/ui/Input';
import { IndexerManagement } from '@/components/admin/indexers/IndexerManagement';
import { FlagConfigRow } from '@/components/admin/FlagConfigRow';
@@ -16,6 +17,7 @@ import type { Settings, SavedIndexerConfig } from '../../lib/types';
interface IndexersTabProps {
settings: Settings;
originalSettings: Settings | null;
indexers: SavedIndexerConfig[];
flagConfigs: IndexerFlagConfig[];
onChange: (settings: Settings) => void;
@@ -27,6 +29,7 @@ interface IndexersTabProps {
export function IndexersTab({
settings,
originalSettings,
indexers,
flagConfigs,
onChange,
@@ -35,11 +38,23 @@ export function IndexersTab({
onValidationChange,
onRefreshIndexers,
}: IndexersTabProps) {
const { testing, testResult, testConnection } = useIndexersSettings({
const {
testing,
testResult,
testConnection,
showConnectionChangeConfirm,
confirmConnectionChange,
cancelConnectionChange,
configuredIndexersCount,
} = useIndexersSettings({
prowlarrUrl: settings.prowlarr.url,
prowlarrApiKey: settings.prowlarr.apiKey,
originalProwlarrUrl: originalSettings?.prowlarr.url ?? '',
originalProwlarrApiKey: originalSettings?.prowlarr.apiKey ?? '',
configuredIndexersCount: indexers.length,
onValidationChange,
onRefreshIndexers,
onClearIndexers: () => onIndexersChange([]),
});
// Auto-load indexers when component mounts if prowlarr is configured
@@ -96,7 +111,7 @@ export function IndexersTab({
placeholder="Enter API key"
/>
<p className="text-sm text-gray-500 dark:text-gray-400 mt-1">
Found in Prowlarr Settings General Security API Key
Found in Prowlarr Settings &rarr; General &rarr; Security &rarr; API Key
</p>
</div>
@@ -178,6 +193,19 @@ export function IndexersTab({
</p>
)}
</div>
{/* Confirmation modal for Prowlarr connection change */}
<ConfirmModal
isOpen={showConnectionChangeConfirm}
onClose={cancelConnectionChange}
onConfirm={confirmConnectionChange}
title="Prowlarr Connection Change"
message={`Changing your Prowlarr connection will remove your ${configuredIndexersCount} configured indexer${configuredIndexersCount === 1 ? '' : 's'}. Indexer IDs are specific to each Prowlarr instance, so existing configurations cannot be preserved. You will need to re-add indexers from the new instance after saving.`}
confirmText="Continue"
cancelText="Cancel"
variant="danger"
isLoading={testing}
/>
</div>
);
}
@@ -5,30 +5,50 @@
'use client';
import { useState } from 'react';
import { useState, useCallback } from 'react';
import { fetchWithAuth } from '@/lib/utils/api';
import type { TestResult } from '../../lib/types';
interface UseIndexersSettingsProps {
prowlarrUrl: string;
prowlarrApiKey: string;
originalProwlarrUrl: string;
originalProwlarrApiKey: string;
configuredIndexersCount: number;
onValidationChange: (isValid: boolean) => void;
onRefreshIndexers?: () => Promise<void>;
onClearIndexers: () => void;
}
export function useIndexersSettings({
prowlarrUrl,
prowlarrApiKey,
originalProwlarrUrl,
originalProwlarrApiKey,
configuredIndexersCount,
onValidationChange,
onRefreshIndexers,
onClearIndexers,
}: UseIndexersSettingsProps) {
const [testing, setTesting] = useState(false);
const [testResult, setTestResult] = useState<TestResult | null>(null);
const [showConnectionChangeConfirm, setShowConnectionChangeConfirm] = useState(false);
/**
* Test Prowlarr connection
* Detect if the Prowlarr URL or API key has changed from the saved values.
* A masked API key (starting with dots) means the user hasn't touched it.
*/
const testConnection = async () => {
const hasConnectionChanged = useCallback((): boolean => {
const urlChanged = prowlarrUrl.trim() !== originalProwlarrUrl.trim();
const apiKeyChanged = !prowlarrApiKey.startsWith('••••') &&
prowlarrApiKey !== originalProwlarrApiKey;
return urlChanged || apiKeyChanged;
}, [prowlarrUrl, prowlarrApiKey, originalProwlarrUrl, originalProwlarrApiKey]);
/**
* Execute the actual Prowlarr connection test
*/
const executeTest = async (shouldClearIndexers: boolean) => {
setTesting(true);
setTestResult(null);
@@ -46,6 +66,14 @@ export function useIndexersSettings({
if (data.success) {
onValidationChange(true);
if (shouldClearIndexers) {
onClearIndexers();
setTestResult({
success: true,
message: `Connected to Prowlarr. Found ${data.indexers?.length || 0} indexers. Previous indexer configurations have been removed — please re-add indexers from the new instance.`,
});
} else {
setTestResult({
success: true,
message: `Connected to Prowlarr. Found ${data.indexers?.length || 0} indexers`,
@@ -55,6 +83,7 @@ export function useIndexersSettings({
if (onRefreshIndexers) {
await onRefreshIndexers();
}
}
} else {
onValidationChange(false);
setTestResult({
@@ -74,9 +103,41 @@ export function useIndexersSettings({
}
};
/**
* Handle test connection click — shows confirmation if credentials changed
* and there are existing configured indexers.
*/
const testConnection = async () => {
if (hasConnectionChanged() && configuredIndexersCount > 0) {
setShowConnectionChangeConfirm(true);
return;
}
await executeTest(false);
};
/**
* User confirmed the credential change — proceed with test and clear indexers on success
*/
const confirmConnectionChange = async () => {
setShowConnectionChangeConfirm(false);
await executeTest(true);
};
/**
* User cancelled the credential change confirmation
*/
const cancelConnectionChange = () => {
setShowConnectionChangeConfirm(false);
};
return {
testing,
testResult,
testConnection,
showConnectionChangeConfirm,
confirmConnectionChange,
cancelConnectionChange,
configuredIndexersCount,
};
}
@@ -164,11 +164,11 @@ export function AudiobookshelfSection({
>
{Object.values(AUDIBLE_REGIONS).map((region) => (
<option key={region.code} value={region.code}>
{region.name}{!region.isEnglish ? ' *' : ''}
{region.name}{region.language !== 'en' ? ' *' : ''}
</option>
))}
</select>
{AUDIBLE_REGIONS[settings.audibleRegion as keyof typeof AUDIBLE_REGIONS]?.isEnglish === false && (
{AUDIBLE_REGIONS[settings.audibleRegion as keyof typeof AUDIBLE_REGIONS]?.language !== 'en' && (
<div className="bg-amber-50 dark:bg-amber-900/20 rounded-lg p-4 border border-amber-200 dark:border-amber-800 mt-2">
<div className="flex gap-3">
<svg
@@ -164,11 +164,11 @@ export function PlexSection({
>
{Object.values(AUDIBLE_REGIONS).map((region) => (
<option key={region.code} value={region.code}>
{region.name}{!region.isEnglish ? ' *' : ''}
{region.name}{region.language !== 'en' ? ' *' : ''}
</option>
))}
</select>
{AUDIBLE_REGIONS[settings.audibleRegion as keyof typeof AUDIBLE_REGIONS]?.isEnglish === false && (
{AUDIBLE_REGIONS[settings.audibleRegion as keyof typeof AUDIBLE_REGIONS]?.language !== 'en' && (
<div className="bg-amber-50 dark:bg-amber-900/20 rounded-lg p-4 border border-amber-200 dark:border-amber-800 mt-2">
<div className="flex gap-3">
<svg
@@ -7,6 +7,7 @@ import { NextRequest, NextResponse } from 'next/server';
import { requireAuth, requireAdmin, AuthenticatedRequest } from '@/lib/middleware/auth';
import { prisma } from '@/lib/db';
import { getEncryptionService } from '@/lib/services/encryption.service';
import { invalidateProwlarrService } from '@/lib/integrations/prowlarr.service';
import { RMABLogger } from '@/lib/utils/logger';
const logger = RMABLogger.create('API.Admin.Settings.Prowlarr');
@@ -42,6 +43,9 @@ export async function PUT(request: NextRequest) {
});
}
// Invalidate cached singleton so background jobs use new credentials
invalidateProwlarrService();
logger.info('Prowlarr settings updated');
return NextResponse.json({
@@ -18,6 +18,8 @@ import { findPlexMatch } from '@/lib/utils/audiobook-matcher';
import { getAudibleService } from '@/lib/integrations/audible.service';
import { RMABLogger } from '@/lib/utils/logger';
import { resolveInteractiveSearchAccess } from '@/lib/utils/permissions';
import { getLanguageForRegion } from '@/lib/constants/language-config';
import type { AudibleRegion } from '@/lib/types/audible';
import {
searchByAsin,
searchByTitle,
@@ -227,6 +229,11 @@ export async function POST(
const format = preferredFormat || 'epub';
const annasBaseUrl = baseUrl || 'https://annas-archive.li';
// Get language code from Audible region config
const region = await configService.getAudibleRegion() as AudibleRegion;
const langConfig = getLanguageForRegion(region);
const languageCode = langConfig.annasArchiveLang;
if (!isAnnasArchiveEnabled && !isIndexerSearchEnabled) {
return NextResponse.json(
{ error: 'No ebook sources enabled. Enable Anna\'s Archive or Indexer Search in settings.' },
@@ -250,7 +257,8 @@ export async function POST(
audiobook.author,
format,
annasBaseUrl,
flaresolverrUrl || undefined
flaresolverrUrl || undefined,
languageCode
).catch((err) => {
logger.error(`Anna's Archive search failed: ${err.message}`);
return null;
@@ -322,7 +330,8 @@ async function searchAnnasArchiveForInteractive(
author: string,
preferredFormat: string,
baseUrl: string,
flaresolverrUrl?: string
flaresolverrUrl?: string,
languageCode: string = 'en'
): Promise<EbookSearchResult[]> {
let md5: string | null = null;
let searchMethod: 'asin' | 'title' = 'title';
@@ -330,7 +339,7 @@ async function searchAnnasArchiveForInteractive(
// Try ASIN search first
if (asin) {
logger.info(`Searching Anna's Archive by ASIN: ${asin}`);
md5 = await searchByAsin(asin, preferredFormat, baseUrl, undefined, flaresolverrUrl);
md5 = await searchByAsin(asin, preferredFormat, baseUrl, undefined, flaresolverrUrl, languageCode);
if (md5) {
searchMethod = 'asin';
logger.info(`Found via ASIN: ${md5}`);
@@ -340,7 +349,7 @@ async function searchAnnasArchiveForInteractive(
// Fallback to title search
if (!md5) {
logger.info(`Searching Anna's Archive by title: "${title}"`);
md5 = await searchByTitle(title, author, preferredFormat, baseUrl, undefined, flaresolverrUrl);
md5 = await searchByTitle(title, author, preferredFormat, baseUrl, undefined, flaresolverrUrl, languageCode);
if (md5) {
logger.info(`Found via title: ${md5}`);
}
@@ -461,6 +470,10 @@ async function searchIndexersForInteractive(
return [];
}
// Get language-specific stop words for ranking
const rankRegion = await configService.getAudibleRegion() as AudibleRegion;
const rankLangConfig = getLanguageForRegion(rankRegion);
// Rank results with ebook scoring
const rankedResults = rankEbookTorrents(allResults, {
title,
@@ -470,6 +483,8 @@ async function searchIndexersForInteractive(
indexerPriorities,
flagConfigs,
requireAuthor: false,
stopWords: rankLangConfig.stopWords,
characterReplacements: rankLangConfig.characterReplacements,
});
// Convert to unified result type
@@ -10,6 +10,8 @@ import { requireAuth, AuthenticatedRequest } from '@/lib/middleware/auth';
import { getProwlarrService } from '@/lib/integrations/prowlarr.service';
import { rankTorrents } from '@/lib/utils/ranking-algorithm';
import { groupIndexersByCategories, getGroupDescription } from '@/lib/utils/indexer-grouping';
import { getLanguageForRegion } from '@/lib/constants/language-config';
import type { AudibleRegion } from '@/lib/types/audible';
import { z } from 'zod';
import { RMABLogger } from '@/lib/utils/logger';
@@ -140,13 +142,19 @@ export async function POST(request: NextRequest) {
logger.info(`Will filter ${belowThreshold.length} results < ${sizeMBThreshold} MB (likely ebooks)`);
}
// Get language-specific stop words for ranking
const region = await configService.getAudibleRegion() as AudibleRegion;
const langConfig = getLanguageForRegion(region);
// Rank torrents using the ranking algorithm with indexer priorities and flag configs
// Note: rankTorrents now filters out results < 20 MB internally
// requireAuthor: false - interactive search, show all results for user decision
const rankedResults = rankTorrents(results, { title, author, durationMinutes }, {
indexerPriorities,
flagConfigs,
requireAuthor: false // Interactive mode - let user decide
requireAuthor: false, // Interactive mode - let user decide
stopWords: langConfig.stopWords,
characterReplacements: langConfig.characterReplacements,
});
// Log filter results
@@ -14,6 +14,8 @@ import { getProwlarrService } from '@/lib/integrations/prowlarr.service';
import { rankEbookTorrents, RankedEbookTorrent } from '@/lib/utils/ranking-algorithm';
import { groupIndexersByCategories, getGroupDescription } from '@/lib/utils/indexer-grouping';
import { RMABLogger } from '@/lib/utils/logger';
import { getLanguageForRegion } from '@/lib/constants/language-config';
import type { AudibleRegion } from '@/lib/types/audible';
import {
searchByAsin,
searchByTitle,
@@ -121,6 +123,11 @@ export async function POST(
const format = preferredFormat || 'epub';
const annasBaseUrl = baseUrl || 'https://annas-archive.li';
// Get language code from Audible region config
const region = await configService.getAudibleRegion() as AudibleRegion;
const langConfig = getLanguageForRegion(region);
const languageCode = langConfig.annasArchiveLang;
if (!isAnnasArchiveEnabled && !isIndexerSearchEnabled) {
return NextResponse.json(
{ error: 'No ebook sources enabled. Enable Anna\'s Archive or Indexer Search in settings.' },
@@ -145,7 +152,8 @@ export async function POST(
audiobook.author,
format,
annasBaseUrl,
flaresolverrUrl || undefined
flaresolverrUrl || undefined,
languageCode
).catch((err) => {
logger.error(`Anna's Archive search failed: ${err.message}`);
return null;
@@ -217,7 +225,8 @@ async function searchAnnasArchiveForInteractive(
author: string,
preferredFormat: string,
baseUrl: string,
flaresolverrUrl?: string
flaresolverrUrl?: string,
languageCode: string = 'en'
): Promise<EbookSearchResult[]> {
let md5: string | null = null;
let searchMethod: 'asin' | 'title' = 'title';
@@ -225,7 +234,7 @@ async function searchAnnasArchiveForInteractive(
// Try ASIN search first
if (asin) {
logger.info(`Searching Anna's Archive by ASIN: ${asin}`);
md5 = await searchByAsin(asin, preferredFormat, baseUrl, undefined, flaresolverrUrl);
md5 = await searchByAsin(asin, preferredFormat, baseUrl, undefined, flaresolverrUrl, languageCode);
if (md5) {
searchMethod = 'asin';
logger.info(`Found via ASIN: ${md5}`);
@@ -235,7 +244,7 @@ async function searchAnnasArchiveForInteractive(
// Fallback to title search
if (!md5) {
logger.info(`Searching Anna's Archive by title: "${title}"`);
md5 = await searchByTitle(title, author, preferredFormat, baseUrl, undefined, flaresolverrUrl);
md5 = await searchByTitle(title, author, preferredFormat, baseUrl, undefined, flaresolverrUrl, languageCode);
if (md5) {
logger.info(`Found via title: ${md5}`);
}
@@ -356,6 +365,10 @@ async function searchIndexersForInteractive(
return [];
}
// Get language-specific stop words for ranking
const rankRegion = await configService.getAudibleRegion() as AudibleRegion;
const rankLangConfig = getLanguageForRegion(rankRegion);
// Rank results with ebook scoring
// Use requireAuthor=false for interactive mode (let user decide)
const rankedResults = rankEbookTorrents(allResults, {
@@ -366,6 +379,8 @@ async function searchIndexersForInteractive(
indexerPriorities,
flagConfigs,
requireAuthor: false,
stopWords: rankLangConfig.stopWords,
characterReplacements: rankLangConfig.characterReplacements,
});
// Log ranking debug info (same format as search-ebook.processor.ts)
@@ -9,6 +9,8 @@ import { prisma } from '@/lib/db';
import { getProwlarrService } from '@/lib/integrations/prowlarr.service';
import { rankTorrents } from '@/lib/utils/ranking-algorithm';
import { groupIndexersByCategories, getGroupDescription } from '@/lib/utils/indexer-grouping';
import { getLanguageForRegion } from '@/lib/constants/language-config';
import type { AudibleRegion } from '@/lib/types/audible';
import { RMABLogger } from '@/lib/utils/logger';
import { resolveInteractiveSearchAccess } from '@/lib/utils/permissions';
@@ -189,6 +191,10 @@ export async function POST(
}
}
// Get language-specific stop words for ranking
const region = await configService.getAudibleRegion() as AudibleRegion;
const langConfig = getLanguageForRegion(region);
// Rank torrents using the ranking algorithm with indexer priorities and flag configs
// Always use the audiobook's title/author for ranking (not custom search query)
// requireAuthor: false - interactive mode, show all results for user decision
@@ -199,7 +205,9 @@ export async function POST(
}, {
indexerPriorities,
flagConfigs,
requireAuthor: false // Interactive mode - let user decide
requireAuthor: false, // Interactive mode - let user decide
stopWords: langConfig.stopWords,
characterReplacements: langConfig.characterReplacements,
});
// No threshold filtering for interactive search - show all results
+72
View File
@@ -0,0 +1,72 @@
/**
* Component: Series Detail API Route
* Documentation: documentation/integrations/audible.md
*/
import { NextRequest, NextResponse } from 'next/server';
import { getCurrentUser } from '@/lib/middleware/auth';
import { RMABLogger } from '@/lib/utils/logger';
import { scrapeSeriesPage } from '@/lib/integrations/audible-series';
import { enrichAudiobooksWithMatches } from '@/lib/utils/audiobook-matcher';
const logger = RMABLogger.create('API.Series.Detail');
/**
* GET /api/series/{asin}
* Fetch series detail: metadata + books (enriched with availability) + similar series
*/
export async function GET(
request: NextRequest,
{ params }: { params: Promise<{ asin: string }> }
) {
try {
const currentUser = getCurrentUser(request);
if (!currentUser) {
return NextResponse.json(
{ error: 'Unauthorized', message: 'Authentication required' },
{ status: 401 }
);
}
const { asin } = await params;
if (!asin || !/^[A-Z0-9]{10}$/.test(asin)) {
return NextResponse.json(
{ error: 'ValidationError', message: 'Valid series ASIN is required' },
{ status: 400 }
);
}
logger.info(`Fetching series detail: ${asin}`);
const detail = await scrapeSeriesPage(asin);
if (!detail) {
return NextResponse.json(
{ error: 'NotFound', message: 'Series not found' },
{ status: 404 }
);
}
// Enrich books with library availability and request status
const userId = currentUser.sub || undefined;
const enrichedBooks = await enrichAudiobooksWithMatches(detail.books, userId);
logger.info(`Series detail complete: "${detail.title}" (${enrichedBooks.length} books)`);
return NextResponse.json({
success: true,
series: {
...detail,
books: enrichedBooks,
},
});
} catch (error) {
logger.error('Failed to fetch series detail', {
error: error instanceof Error ? error.message : String(error),
});
return NextResponse.json(
{ error: 'FetchError', message: 'Failed to fetch series details' },
{ status: 500 }
);
}
}
+57
View File
@@ -0,0 +1,57 @@
/**
* Component: Series Search API Route
* Documentation: documentation/integrations/audible.md
*/
import { NextRequest, NextResponse } from 'next/server';
import { getCurrentUser } from '@/lib/middleware/auth';
import { RMABLogger } from '@/lib/utils/logger';
import { searchForSeries } from '@/lib/integrations/audible-series';
const logger = RMABLogger.create('API.Series.Search');
/**
* GET /api/series/search?q=game+of+thrones
* Search for audiobook series on Audible, de-duplicate, and return enriched summaries
*/
export async function GET(request: NextRequest) {
try {
// Require authentication
const currentUser = getCurrentUser(request);
if (!currentUser) {
return NextResponse.json(
{ error: 'Unauthorized', message: 'Authentication required' },
{ status: 401 }
);
}
const query = request.nextUrl.searchParams.get('q');
if (!query || query.trim().length === 0) {
return NextResponse.json(
{ error: 'ValidationError', message: 'Search query is required' },
{ status: 400 }
);
}
logger.info(`Searching series: "${query}"`);
const series = await searchForSeries(query.trim());
logger.info(`Series search complete: "${query}" -> ${series.length} results`);
return NextResponse.json({
success: true,
series,
query: query.trim(),
});
} catch (error) {
logger.error('Failed to search series', {
error: error instanceof Error ? error.message : String(error),
});
return NextResponse.json(
{ error: 'SearchError', message: 'Failed to search series' },
{ status: 500 }
);
}
}
+117
View File
@@ -0,0 +1,117 @@
/**
* Component: Series Detail Page
* Documentation: documentation/frontend/components.md
*/
'use client';
import { use, useCallback } from 'react';
import { useRouter, useSearchParams } from 'next/navigation';
import { Header } from '@/components/layout/Header';
import { AudiobookGrid } from '@/components/audiobooks/AudiobookGrid';
import { SeriesDetailCard, SeriesDetailSkeleton } from '@/components/series/SeriesDetailCard';
import { SimilarSeriesRow, SimilarSeriesSkeleton } from '@/components/series/SimilarSeriesRow';
import { useSeriesDetail } from '@/lib/hooks/useSeries';
import { ProtectedRoute } from '@/components/auth/ProtectedRoute';
import { CardSizeControls } from '@/components/ui/CardSizeControls';
import { SquareCoversToggle } from '@/components/ui/SquareCoversToggle';
import { usePreferences } from '@/contexts/PreferencesContext';
export default function SeriesDetailPage({
params,
}: {
params: Promise<{ asin: string }>;
}) {
const { asin } = use(params);
const router = useRouter();
const searchParams = useSearchParams();
const fromSeriesTitle = searchParams.get('from');
const { series, isLoading: seriesLoading } = useSeriesDetail(asin);
const { cardSize, setCardSize, squareCovers, setSquareCovers } = usePreferences();
const handleBack = useCallback(() => {
// Use browser back if we came from within the app, otherwise fallback to /series
if (window.history.length > 1) {
router.back();
} else {
router.push('/series');
}
}, [router]);
return (
<ProtectedRoute>
<div className="min-h-screen">
<Header />
<main className="container mx-auto px-4 py-6 sm:py-8 max-w-7xl space-y-8">
{/* Back navigation */}
<button
onClick={handleBack}
className="inline-flex items-center gap-1.5 text-sm text-gray-500 dark:text-gray-400 hover:text-indigo-600 dark:hover:text-indigo-400 transition-colors"
>
<svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M15 19l-7-7 7-7" />
</svg>
{fromSeriesTitle ? `Back to ${fromSeriesTitle}` : 'Back to Series'}
</button>
{/* Series Detail Card */}
{seriesLoading ? (
<SeriesDetailSkeleton squareCovers={squareCovers} />
) : series ? (
<SeriesDetailCard series={series} squareCovers={squareCovers} />
) : (
<div className="text-center py-16 space-y-4">
<svg className="mx-auto h-16 w-16 text-gray-400" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-2.5L13.732 4.5c-.77-.833-1.964-.833-2.732 0L4.082 16.5c-.77.833.192 2.5 1.732 2.5z" />
</svg>
<p className="text-xl text-gray-600 dark:text-gray-400">Series not found</p>
</div>
)}
{/* Similar Series */}
{seriesLoading ? (
<SimilarSeriesSkeleton squareCovers={squareCovers} />
) : series && series.similarSeries.length > 0 ? (
<SimilarSeriesRow series={series.similarSeries} currentSeriesTitle={series.title} squareCovers={squareCovers} />
) : null}
{/* Books Section */}
{series && (
<div className="space-y-6">
{/* Sticky Books Header */}
<div className="sticky top-14 sm:top-16 z-30">
<div className="bg-white/90 dark:bg-gray-800/90 backdrop-blur-md rounded-2xl px-4 sm:px-6 py-3 border border-gray-200/50 dark:border-gray-700/50 shadow-sm">
<div className="flex items-center gap-3">
<div className="w-1 h-6 bg-gradient-to-b from-blue-500 to-purple-500 rounded-full" />
<h2 className="text-xl sm:text-2xl font-bold text-gray-900 dark:text-gray-100 truncate">
Books in Series
</h2>
{series.books.length > 0 && (
<span className="text-sm text-gray-600 dark:text-gray-400 hidden sm:inline whitespace-nowrap">
({series.books.length} title{series.books.length !== 1 ? 's' : ''})
</span>
)}
<div className="ml-auto flex items-center gap-1">
<SquareCoversToggle enabled={squareCovers} onToggle={setSquareCovers} />
<CardSizeControls size={cardSize} onSizeChange={setCardSize} />
</div>
</div>
</div>
</div>
{/* Books Grid */}
<AudiobookGrid
audiobooks={series.books}
isLoading={seriesLoading}
emptyMessage={`No books found for ${series.title}`}
cardSize={cardSize}
squareCovers={squareCovers}
/>
</div>
)}
</main>
</div>
</ProtectedRoute>
);
}
+179
View File
@@ -0,0 +1,179 @@
/**
* Component: Series Page
* Documentation: documentation/frontend/components.md
*/
'use client';
import { Suspense, useState, useEffect, useCallback } from 'react';
import { useSearchParams, useRouter } from 'next/navigation';
import { Header } from '@/components/layout/Header';
import { SeriesGrid } from '@/components/series/SeriesGrid';
import { useSeriesSearch } from '@/lib/hooks/useSeries';
import { ProtectedRoute } from '@/components/auth/ProtectedRoute';
import { CardSizeControls } from '@/components/ui/CardSizeControls';
import { SquareCoversToggle } from '@/components/ui/SquareCoversToggle';
import { usePreferences } from '@/contexts/PreferencesContext';
function SeriesPageContent() {
const searchParams = useSearchParams();
const router = useRouter();
const initialQuery = searchParams.get('q') || '';
const [query, setQuery] = useState(initialQuery);
const [debouncedQuery, setDebouncedQuery] = useState(initialQuery);
const { cardSize, setCardSize, squareCovers, setSquareCovers } = usePreferences();
// Debounce search query and sync to URL
useEffect(() => {
const timer = setTimeout(() => {
setDebouncedQuery(query);
// Update URL without adding history entries
const trimmed = query.trim();
if (trimmed) {
router.replace(`/series?q=${encodeURIComponent(trimmed)}`, { scroll: false });
} else {
router.replace('/series', { scroll: false });
}
}, 500);
return () => clearTimeout(timer);
}, [query, router]);
const { series, isLoading } = useSeriesSearch(debouncedQuery);
const handleSearch = useCallback((e: React.FormEvent) => {
e.preventDefault();
}, []);
return (
<ProtectedRoute>
<div className="min-h-screen">
<Header />
<main className="container mx-auto px-4 py-8 max-w-7xl space-y-8">
{/* Page Header */}
<div className="text-center space-y-4">
<h1 className="text-4xl font-bold text-gray-900 dark:text-gray-100">
Browse Series
</h1>
<p className="text-gray-600 dark:text-gray-400">
Search for your favorite audiobook series
</p>
</div>
{/* Search Form */}
<form onSubmit={handleSearch} className="max-w-3xl mx-auto">
<div className="relative">
<div className="absolute inset-y-0 left-0 pl-4 flex items-center pointer-events-none">
<svg
className="h-5 w-5 text-gray-400"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={2}
d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z"
/>
</svg>
</div>
<input
type="text"
value={query}
onChange={(e) => setQuery(e.target.value)}
placeholder="Search by series name..."
className="w-full pl-12 pr-12 py-4 text-lg border-2 border-gray-300 dark:border-gray-600 rounded-lg focus:ring-2 focus:ring-blue-500 focus:border-transparent bg-white dark:bg-gray-800 text-gray-900 dark:text-gray-100 placeholder-gray-400"
autoFocus
/>
{query && (
<button
type="button"
onClick={() => setQuery('')}
className="absolute inset-y-0 right-0 pr-4 flex items-center text-gray-400 hover:text-gray-600 dark:hover:text-gray-200"
>
<svg className="h-5 w-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={2}
d="M6 18L18 6M6 6l12 12"
/>
</svg>
</button>
)}
</div>
</form>
{/* Results */}
{debouncedQuery ? (
<div className="space-y-6">
{/* Sticky Results Header */}
<div className="sticky top-14 sm:top-16 z-30 mb-4 sm:mb-6">
<div className="bg-white/90 dark:bg-gray-800/90 backdrop-blur-md rounded-2xl px-4 sm:px-6 py-3 border border-gray-200/50 dark:border-gray-700/50 shadow-sm">
<div className="flex items-center gap-3">
<div className="w-1 h-6 bg-gradient-to-b from-emerald-500 to-teal-500 rounded-full" />
<h2 className="text-xl sm:text-2xl font-bold text-gray-900 dark:text-gray-100 truncate">
Series
</h2>
{!isLoading && series.length > 0 && (
<span className="text-sm text-gray-600 dark:text-gray-400 hidden sm:inline whitespace-nowrap">
({series.length} result{series.length !== 1 ? 's' : ''})
</span>
)}
<div className="ml-auto flex items-center gap-1">
<SquareCoversToggle enabled={squareCovers} onToggle={setSquareCovers} />
<CardSizeControls size={cardSize} onSizeChange={setCardSize} />
</div>
</div>
</div>
</div>
{/* Series Grid */}
<SeriesGrid
series={series}
isLoading={!!isLoading}
emptyMessage={`No series found for "${debouncedQuery}"`}
cardSize={cardSize}
squareCovers={squareCovers}
/>
</div>
) : (
/* Empty State */
<div className="text-center py-16 space-y-4">
<svg
className="mx-auto h-16 w-16 text-gray-400"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={2}
d="M12 6.042A8.967 8.967 0 006 3.75c-1.052 0-2.062.18-3 .512v14.25A8.987 8.987 0 016 18c2.305 0 4.408.867 6 2.292m0-14.25a8.966 8.966 0 016-2.292c1.052 0 2.062.18 3 .512v14.25A8.987 8.987 0 0018 18a8.967 8.967 0 00-6 2.292m0-14.25v14.25"
/>
</svg>
<p className="text-xl text-gray-600 dark:text-gray-400">
Start typing to search for series
</p>
<p className="text-sm text-gray-500 dark:text-gray-500">
Search by series name to discover audiobook collections
</p>
</div>
)}
</main>
</div>
</ProtectedRoute>
);
}
export default function SeriesPage() {
return (
<Suspense>
<SeriesPageContent />
</Suspense>
);
}
+2 -2
View File
@@ -115,11 +115,11 @@ export function BackendSelectionStep({
>
{Object.values(AUDIBLE_REGIONS).map((region) => (
<option key={region.code} value={region.code}>
{region.name}{!region.isEnglish ? ' *' : ''}
{region.name}{region.language !== 'en' ? ' *' : ''}
</option>
))}
</select>
{AUDIBLE_REGIONS[audibleRegion]?.isEnglish === false && (
{AUDIBLE_REGIONS[audibleRegion]?.language !== 'en' && (
<div className="bg-amber-50 dark:bg-amber-900/20 rounded-lg p-4 border border-amber-200 dark:border-amber-800 mt-2">
<div className="flex gap-3">
<svg
@@ -253,7 +253,7 @@ export function DownloadClientManagement({
<h3 className="text-lg font-semibold text-gray-900 dark:text-gray-100 mb-3">
Add Download Client
</h3>
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-5 gap-4">
{/* qBittorrent Card */}
<div className={`bg-white dark:bg-gray-800 rounded-lg shadow-md border border-gray-200 dark:border-gray-700 p-6${hasTorrentClient ? ' opacity-50' : ''}`}>
<div className="flex items-start justify-between mb-3">
@@ -316,6 +316,37 @@ export function DownloadClientManagement({
)}
</div>
{/* RDT-Client Card */}
<div className={`bg-white dark:bg-gray-800 rounded-lg shadow-md border border-gray-200 dark:border-gray-700 p-6${hasTorrentClient ? ' opacity-50' : ''}`}>
<div className="flex items-start justify-between mb-3">
<div>
<h4 className="text-base font-semibold text-gray-900 dark:text-gray-100 mb-1">
RDT-Client
</h4>
<p className="text-sm text-gray-600 dark:text-gray-400">
Torrent / Debrid
</p>
</div>
<span className="inline-block text-xs px-2 py-1 rounded bg-teal-100 dark:bg-teal-900/30 text-teal-700 dark:text-teal-300 font-medium">
Torrent
</span>
</div>
{hasTorrentClient ? (
<div className="text-sm text-gray-500 dark:text-gray-400">
Protocol already configured
</div>
) : (
<Button
onClick={() => handleAddClient('rdtclient')}
variant="primary"
size="sm"
disabled={loading}
>
Add RDT-Client
</Button>
)}
</div>
{/* SABnzbd Card */}
<div className={`bg-white dark:bg-gray-800 rounded-lg shadow-md border border-gray-200 dark:border-gray-700 p-6${hasUsenetClient ? ' opacity-50' : ''}`}>
<div className="flex items-start justify-between mb-3">
@@ -286,7 +286,7 @@ export function DownloadClientModal({
remotePath: remotePathMappingEnabled ? remotePath : undefined,
localPath: remotePathMappingEnabled ? localPath : undefined,
category,
customPath: sanitizedCustomPath || undefined,
customPath: sanitizedCustomPath,
postImportCategory,
};
@@ -338,7 +338,7 @@ export function DownloadClientModal({
<Input
value={url}
onChange={(e) => setUrl(e.target.value)}
placeholder={type === 'transmission' ? 'http://localhost:9091' : type === 'qbittorrent' ? 'http://localhost:8080' : type === 'nzbget' ? 'http://localhost:6789' : 'http://localhost:8081'}
placeholder={type === 'rdtclient' ? 'http://localhost:6500' : type === 'transmission' ? 'http://localhost:9091' : type === 'qbittorrent' ? 'http://localhost:8080' : type === 'nzbget' ? 'http://localhost:6789' : 'http://localhost:8081'}
error={errors.url}
/>
<p className="mt-1 text-xs text-gray-500 dark:text-gray-400">
@@ -307,6 +307,24 @@ export function AudiobookDetailsModal({
Narrated by {audiobook.narrator}
</p>
)}
{audiobook.series && (
<p className="mt-1 text-sm text-gray-500 dark:text-gray-400">
{audiobook.seriesAsin ? (
<Link
href={`/series/${audiobook.seriesAsin}`}
onClick={(e) => {
e.stopPropagation();
onClose();
}}
className="hover:text-emerald-600 dark:hover:text-emerald-400 transition-colors"
>
{audiobook.series}{audiobook.seriesPart ? `, Book ${audiobook.seriesPart}` : ''}
</Link>
) : (
<span>{audiobook.series}{audiobook.seriesPart ? `, Book ${audiobook.seriesPart}` : ''}</span>
)}
</p>
)}
{/* Status Badge */}
{status.type !== 'none' && (
+13
View File
@@ -166,6 +166,12 @@ export function Header() {
>
Authors
</Link>
<Link
href="/series"
className="text-gray-700 dark:text-gray-300 hover:text-blue-600 dark:hover:text-blue-400 transition-colors"
>
Series
</Link>
{showBookDate && (
<Link
href="/bookdate"
@@ -277,6 +283,13 @@ export function Header() {
>
Authors
</Link>
<Link
href="/series"
onClick={() => setShowMobileMenu(false)}
className="px-3 py-2 text-gray-700 dark:text-gray-300 hover:bg-gray-100 dark:hover:bg-gray-700 rounded-md transition-colors"
>
Series
</Link>
{showBookDate && (
<Link
href="/bookdate"
+153
View File
@@ -0,0 +1,153 @@
/**
* Component: Series Card
* Documentation: documentation/frontend/components.md
*
* Premium "Cover First" design - metadata integrated into the cover overlay.
* Rating badge top-left, book count top-right, tags in bottom gradient overlay.
* Only the title lives below the cover, ensuring consistent row heights in the grid.
*/
'use client';
import React from 'react';
import Image from 'next/image';
import Link from 'next/link';
import { SeriesSummary } from '@/lib/hooks/useSeries';
interface SeriesCardProps {
series: SeriesSummary;
squareCovers?: boolean;
}
export function SeriesCard({ series, squareCovers = false }: SeriesCardProps) {
const visibleTags = series.tags.slice(0, 2);
const hasTags = visibleTags.length > 0;
const hasRating = series.rating != null && series.rating > 0;
return (
<Link
href={`/series/${series.asin}`}
className="group outline-none focus-visible:ring-2 focus-visible:ring-emerald-500 focus-visible:ring-offset-2 focus-visible:ring-offset-transparent rounded-2xl block"
aria-label={`View ${series.title} series`}
>
{/* Cover Container — The Hero */}
<div
className={`
relative overflow-hidden rounded-xl
w-full ${squareCovers ? 'aspect-square' : 'aspect-[2/3]'}
shadow-lg shadow-black/20 dark:shadow-black/40
group-hover:shadow-xl group-hover:shadow-black/30 dark:group-hover:shadow-black/55
transform group-hover:scale-[1.02] group-hover:-translate-y-0.5
transition-all duration-300 ease-out
`}
>
{/* Cover Art or Fallback */}
{series.coverArtUrl ? (
<Image
src={series.coverArtUrl}
alt=""
fill
className="object-cover"
sizes="(max-width: 640px) 50vw, (max-width: 1024px) 33vw, 20vw"
/>
) : (
<div className="absolute inset-0 bg-gradient-to-br from-emerald-600 to-teal-800 dark:from-emerald-700 dark:to-teal-900 flex items-center justify-center">
<svg
className="w-1/3 h-1/3 text-white/40"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={1.2}
d="M12 6.042A8.967 8.967 0 006 3.75c-1.052 0-2.062.18-3 .512v14.25A8.987 8.987 0 016 18c2.305 0 4.408.867 6 2.292m0-14.25a8.966 8.966 0 016-2.292c1.052 0 2.062.18 3 .512v14.25A8.987 8.987 0 0018 18a8.967 8.967 0 00-6 2.292m0-14.25v14.25"
/>
</svg>
</div>
)}
{/* Top-row badges — Rating (left) + Book count (right) */}
{/* Rating Badge — top-left, matches AudiobookCard pattern exactly */}
{hasRating && (
<div className="
absolute top-2.5 left-2.5
flex items-center gap-1 px-2 py-1
rounded-lg bg-black/50 backdrop-blur-md
text-white text-xs font-medium
transition-opacity duration-300 group-hover:opacity-0
">
<svg className="w-3.5 h-3.5 text-amber-400 shrink-0" viewBox="0 0 20 20" fill="currentColor">
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.175 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81h3.461a1 1 0 00.951-.69l1.07-3.292z" />
</svg>
<span>{series.rating!.toFixed(1)}</span>
</div>
)}
{/* Book count badge — top-right */}
{series.bookCount > 0 && (
<div className="
absolute top-2.5 right-2.5
px-2 py-1
text-[11px] font-bold rounded-lg
bg-black/50 backdrop-blur-md
text-white
transition-opacity duration-300 group-hover:opacity-0
">
{series.bookCount} {series.bookCount === 1 ? 'Book' : 'Books'}
</div>
)}
{/* Bottom gradient overlay — always present, deepens on hover */}
<div className={`
absolute inset-x-0 bottom-0
transition-all duration-300
${hasTags
? 'h-20 bg-gradient-to-t from-black/75 via-black/30 to-transparent group-hover:h-24 group-hover:from-black/85'
: 'h-10 bg-gradient-to-t from-black/40 to-transparent opacity-0 group-hover:opacity-100'
}
`} />
{/* Tag pills — pinned to bottom of cover, inside gradient */}
{hasTags && (
<div className="
absolute inset-x-0 bottom-0
flex items-end gap-1.5 p-2.5
pointer-events-none
">
{visibleTags.map(tag => (
<span
key={tag}
className="
inline-block px-2.5 py-0.5
text-[10px] font-medium
rounded-full
bg-black/30 backdrop-blur-md
text-white/90
ring-1 ring-white/15
transition-opacity duration-300
"
>
{tag}
</span>
))}
</div>
)}
</div>
{/* Below-cover: title only — fixed, predictable height across all cards */}
<div className="mt-2.5 px-0.5">
<h3 className="
font-semibold text-[14px] leading-snug
text-gray-900 dark:text-gray-100
line-clamp-2
group-hover:text-emerald-600 dark:group-hover:text-emerald-400
transition-colors duration-200
">
{series.title}
</h3>
</div>
</Link>
);
}
+164
View File
@@ -0,0 +1,164 @@
/**
* Component: Series Detail Card
* Documentation: documentation/frontend/components.md
*
* Hero section for the series detail page with rectangular cover image,
* title, book count, rating, collapsible description, and tag pills.
*/
'use client';
import React, { useState } from 'react';
import Image from 'next/image';
import { SeriesDetail } from '@/lib/hooks/useSeries';
interface SeriesDetailCardProps {
series: SeriesDetail;
squareCovers?: boolean;
}
export function SeriesDetailCard({ series, squareCovers = false }: SeriesDetailCardProps) {
const [expanded, setExpanded] = useState(false);
const hasLongDescription = (series.description?.length || 0) > 300;
return (
<div className="flex flex-col sm:flex-row items-center sm:items-start gap-6 sm:gap-8">
{/* Rectangular Cover */}
<div className="flex-shrink-0">
<div className={`relative w-36 sm:w-44 lg:w-52 ${squareCovers ? 'aspect-square' : 'aspect-[2/3]'} rounded-xl overflow-hidden shadow-xl shadow-black/20 dark:shadow-black/40`}>
{series.books[0]?.coverArtUrl ? (
<Image
src={series.books[0].coverArtUrl}
alt={series.title}
fill
className="object-cover"
sizes="(max-width: 640px) 144px, (max-width: 1024px) 176px, 208px"
priority
/>
) : (
<div className="absolute inset-0 bg-gradient-to-br from-emerald-100 to-teal-200 dark:from-emerald-900 dark:to-teal-900 flex items-center justify-center">
<svg className="w-1/3 h-1/3 text-emerald-400 dark:text-emerald-300" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M12 6.042A8.967 8.967 0 006 3.75c-1.052 0-2.062.18-3 .512v14.25A8.987 8.987 0 016 18c2.305 0 4.408.867 6 2.292m0-14.25a8.966 8.966 0 016-2.292c1.052 0 2.062.18 3 .512v14.25A8.987 8.987 0 0018 18a8.967 8.967 0 00-6 2.292m0-14.25v14.25" />
</svg>
</div>
)}
</div>
</div>
{/* Series Info */}
<div className="flex-1 min-w-0 text-center sm:text-left">
<h1 className="text-3xl sm:text-4xl font-bold text-gray-900 dark:text-gray-100">
{series.title}
</h1>
{/* Meta row: book count + rating */}
<div className="mt-3 flex flex-wrap items-center justify-center sm:justify-start gap-3">
{series.bookCount > 0 && (
<span className="inline-flex items-center gap-1.5 px-3 py-1 text-sm font-medium rounded-full bg-emerald-50 dark:bg-emerald-900/30 text-emerald-600 dark:text-emerald-300">
<svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 6.042A8.967 8.967 0 006 3.75c-1.052 0-2.062.18-3 .512v14.25A8.987 8.987 0 016 18c2.305 0 4.408.867 6 2.292m0-14.25a8.966 8.966 0 016-2.292c1.052 0 2.062.18 3 .512v14.25A8.987 8.987 0 0018 18a8.967 8.967 0 00-6 2.292m0-14.25v14.25" />
</svg>
{series.bookCount} Book{series.bookCount !== 1 ? 's' : ''}
</span>
)}
{series.rating != null && series.rating > 0 && (
<span className="inline-flex items-center gap-1 text-sm text-gray-600 dark:text-gray-400">
<svg className="w-4 h-4 text-amber-400" fill="currentColor" viewBox="0 0 20 20">
<path d="M9.049 2.927c.3-.921 1.603-.921 1.902 0l1.07 3.292a1 1 0 00.95.69h3.462c.969 0 1.371 1.24.588 1.81l-2.8 2.034a1 1 0 00-.364 1.118l1.07 3.292c.3.921-.755 1.688-1.54 1.118l-2.8-2.034a1 1 0 00-1.175 0l-2.8 2.034c-.784.57-1.838-.197-1.539-1.118l1.07-3.292a1 1 0 00-.364-1.118L2.98 8.72c-.783-.57-.38-1.81.588-1.81h3.461a1 1 0 00.951-.69l1.07-3.292z" />
</svg>
{series.rating.toFixed(1)}
{series.ratingCount != null && series.ratingCount > 0 && (
<span className="text-gray-400 dark:text-gray-500">
({series.ratingCount.toLocaleString()})
</span>
)}
</span>
)}
</div>
{/* Tag Pills */}
{series.tags.length > 0 && (
<div className="mt-3 flex flex-wrap justify-center sm:justify-start gap-2">
{series.tags.map(tag => (
<span
key={tag}
className="inline-block px-3 py-1 text-xs font-medium rounded-full bg-indigo-50 dark:bg-indigo-900/30 text-indigo-600 dark:text-indigo-300"
>
{tag}
</span>
))}
</div>
)}
{/* Audible Link */}
{series.audibleUrl && (
<a
href={series.audibleUrl}
target="_blank"
rel="noopener noreferrer"
className="mt-3 inline-flex items-center gap-1.5 text-sm text-gray-500 dark:text-gray-400 hover:text-indigo-600 dark:hover:text-indigo-400 transition-colors"
>
View on Audible
<svg className="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M10 6H6a2 2 0 00-2 2v10a2 2 0 002 2h10a2 2 0 002-2v-4M14 4h6m0 0v6m0-6L10 14" />
</svg>
</a>
)}
{/* Description */}
{series.description && (
<div className="mt-4">
<p
className={`text-sm sm:text-base text-gray-600 dark:text-gray-400 leading-relaxed whitespace-pre-line ${
!expanded && hasLongDescription ? 'line-clamp-4' : ''
}`}
>
{series.description}
</p>
{hasLongDescription && (
<button
onClick={() => setExpanded(!expanded)}
className="mt-1 text-sm font-medium text-indigo-600 dark:text-indigo-400 hover:text-indigo-800 dark:hover:text-indigo-300 transition-colors"
>
{expanded ? 'Show less' : 'Read more'}
</button>
)}
</div>
)}
</div>
</div>
);
}
export function SeriesDetailSkeleton({ squareCovers = false }: { squareCovers?: boolean }) {
return (
<div className="animate-pulse flex flex-col sm:flex-row items-center sm:items-start gap-6 sm:gap-8">
{/* Cover skeleton */}
<div className="flex-shrink-0">
<div className={`w-36 sm:w-44 lg:w-52 ${squareCovers ? 'aspect-square' : 'aspect-[2/3]'} rounded-xl bg-gradient-to-br from-gray-200 to-gray-300 dark:from-gray-700 dark:to-gray-800 relative overflow-hidden`}>
<div className="absolute inset-0 -translate-x-full animate-[shimmer_2s_infinite] bg-gradient-to-r from-transparent via-white/20 to-transparent" />
</div>
</div>
{/* Info skeleton */}
<div className="flex-1 min-w-0 text-center sm:text-left space-y-4">
<div className="h-9 bg-gray-200 dark:bg-gray-700 rounded-lg w-64 mx-auto sm:mx-0" />
<div className="flex gap-2 justify-center sm:justify-start">
<div className="h-7 w-24 bg-gray-200 dark:bg-gray-700 rounded-full" />
<div className="h-7 w-20 bg-gray-200 dark:bg-gray-700 rounded-full" />
</div>
<div className="flex gap-2 justify-center sm:justify-start">
<div className="h-6 w-20 bg-gray-200 dark:bg-gray-700 rounded-full" />
<div className="h-6 w-24 bg-gray-200 dark:bg-gray-700 rounded-full" />
<div className="h-6 w-16 bg-gray-200 dark:bg-gray-700 rounded-full" />
</div>
<div className="space-y-2">
<div className="h-4 bg-gray-200 dark:bg-gray-700 rounded w-full" />
<div className="h-4 bg-gray-200 dark:bg-gray-700 rounded w-5/6" />
<div className="h-4 bg-gray-200 dark:bg-gray-700 rounded w-4/6" />
</div>
</div>
</div>
);
}
+98
View File
@@ -0,0 +1,98 @@
/**
* Component: Series Grid
* Documentation: documentation/frontend/components.md
*
* Grid layout for series cards with loading skeletons and empty state.
* Uses the same responsive column system as AudiobookGrid since
* series cards use rectangular (2:3) aspect ratios like book covers.
*/
'use client';
import React from 'react';
import { SeriesCard } from './SeriesCard';
import { SeriesSummary } from '@/lib/hooks/useSeries';
interface SeriesGridProps {
series: SeriesSummary[];
isLoading?: boolean;
emptyMessage?: string;
cardSize?: number;
squareCovers?: boolean;
}
function getGridClasses(size: number): string {
const sizeMap: Record<number, string> = {
1: 'grid-cols-4 sm:grid-cols-5 md:grid-cols-6 lg:grid-cols-8 xl:grid-cols-10',
2: 'grid-cols-3 sm:grid-cols-4 md:grid-cols-5 lg:grid-cols-7 xl:grid-cols-9',
3: 'grid-cols-3 sm:grid-cols-4 md:grid-cols-5 lg:grid-cols-6 xl:grid-cols-8',
4: 'grid-cols-2 sm:grid-cols-3 md:grid-cols-4 lg:grid-cols-5 xl:grid-cols-7',
5: 'grid-cols-2 sm:grid-cols-3 md:grid-cols-4 lg:grid-cols-5 xl:grid-cols-6',
6: 'grid-cols-2 sm:grid-cols-3 md:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5',
7: 'grid-cols-2 sm:grid-cols-2 md:grid-cols-3 lg:grid-cols-4',
8: 'grid-cols-1 sm:grid-cols-2 md:grid-cols-3',
9: 'grid-cols-1 sm:grid-cols-2',
};
return sizeMap[size] || sizeMap[5];
}
export function SeriesGrid({
series,
isLoading = false,
emptyMessage = 'No series found',
cardSize = 5,
squareCovers = false,
}: SeriesGridProps) {
const gridClasses = getGridClasses(cardSize);
if (isLoading) {
return (
<div className={`grid ${gridClasses} gap-4 sm:gap-5 lg:gap-6`}>
{Array.from({ length: 10 }).map((_, i) => (
<SeriesSkeletonCard key={i} index={i} squareCovers={squareCovers} />
))}
</div>
);
}
if (series.length === 0) {
return (
<div className="flex flex-col items-center justify-center py-20 text-center">
<div className="w-20 h-20 rounded-2xl bg-gray-100 dark:bg-gray-800 flex items-center justify-center mb-6">
<svg className="w-10 h-10 text-gray-400 dark:text-gray-500" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M12 6.042A8.967 8.967 0 006 3.75c-1.052 0-2.062.18-3 .512v14.25A8.987 8.987 0 016 18c2.305 0 4.408.867 6 2.292m0-14.25a8.966 8.966 0 016-2.292c1.052 0 2.062.18 3 .512v14.25A8.987 8.987 0 0018 18a8.967 8.967 0 00-6 2.292m0-14.25v14.25" />
</svg>
</div>
<p className="text-gray-500 dark:text-gray-400 text-lg">{emptyMessage}</p>
</div>
);
}
return (
<div className={`grid ${gridClasses} gap-4 sm:gap-5 lg:gap-6`}>
{series.map(s => (
<SeriesCard key={s.asin} series={s} squareCovers={squareCovers} />
))}
</div>
);
}
function SeriesSkeletonCard({ index = 0, squareCovers = false }: { index?: number; squareCovers?: boolean }) {
return (
<div
className="animate-pulse"
style={{ animationDelay: `${index * 50}ms` }}
>
{/* Rectangular cover skeleton */}
<div className={`relative overflow-hidden rounded-xl w-full ${squareCovers ? 'aspect-square' : 'aspect-[2/3]'} bg-gradient-to-br from-gray-200 to-gray-300 dark:from-gray-700 dark:to-gray-800`}>
<div className="absolute inset-0 -translate-x-full animate-[shimmer_2s_infinite] bg-gradient-to-r from-transparent via-white/20 to-transparent" />
</div>
{/* Text skeleton */}
<div className="mt-3 px-1 flex flex-col items-center space-y-2">
<div className="h-4 bg-gray-200 dark:bg-gray-700 rounded-lg w-4/5" />
<div className="h-3 bg-gray-200 dark:bg-gray-700 rounded-lg w-3/5" />
</div>
</div>
);
}
+169
View File
@@ -0,0 +1,169 @@
/**
* Component: Similar Series Row
* Documentation: documentation/frontend/components.md
*
* Horizontal scrollable carousel of similar series cards.
* Desktop: left/right nav arrows. Mobile: drag-to-scroll.
* Each card navigates to the series detail page.
*/
'use client';
import React, { useRef, useState, useEffect, useCallback } from 'react';
import Image from 'next/image';
import Link from 'next/link';
import { SimilarSeries } from '@/lib/hooks/useSeries';
interface SimilarSeriesRowProps {
series: SimilarSeries[];
currentSeriesTitle?: string;
squareCovers?: boolean;
}
export function SimilarSeriesRow({ series, currentSeriesTitle, squareCovers = false }: SimilarSeriesRowProps) {
const scrollRef = useRef<HTMLDivElement>(null);
const [canScrollLeft, setCanScrollLeft] = useState(false);
const [canScrollRight, setCanScrollRight] = useState(false);
const checkScroll = useCallback(() => {
const el = scrollRef.current;
if (!el) return;
setCanScrollLeft(el.scrollLeft > 4);
setCanScrollRight(el.scrollLeft < el.scrollWidth - el.clientWidth - 4);
}, []);
useEffect(() => {
checkScroll();
const el = scrollRef.current;
if (!el) return;
el.addEventListener('scroll', checkScroll, { passive: true });
const observer = new ResizeObserver(checkScroll);
observer.observe(el);
return () => {
el.removeEventListener('scroll', checkScroll);
observer.disconnect();
};
}, [checkScroll, series]);
const scroll = (direction: 'left' | 'right') => {
const el = scrollRef.current;
if (!el) return;
const scrollAmount = el.clientWidth * 0.7;
el.scrollBy({
left: direction === 'left' ? -scrollAmount : scrollAmount,
behavior: 'smooth',
});
};
if (series.length === 0) return null;
return (
<div className="space-y-3">
<div className="flex items-center gap-3">
<div className="w-1 h-6 bg-gradient-to-b from-emerald-500 to-teal-500 rounded-full" />
<h2 className="text-xl sm:text-2xl font-bold text-gray-900 dark:text-gray-100">
Similar Series
</h2>
<span className="text-sm text-gray-500 dark:text-gray-400">
({series.length})
</span>
</div>
<div className="relative group">
{/* Left arrow */}
{canScrollLeft && (
<button
onClick={() => scroll('left')}
className="hidden md:flex absolute left-0 top-1/2 -translate-y-1/2 -translate-x-3 z-10 w-10 h-10 bg-white dark:bg-gray-800 rounded-full shadow-lg items-center justify-center text-gray-600 dark:text-gray-300 hover:bg-gray-50 dark:hover:bg-gray-700 transition-all opacity-0 group-hover:opacity-100"
aria-label="Scroll left"
>
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M15 19l-7-7 7-7" />
</svg>
</button>
)}
{/* Scrollable row */}
<div
ref={scrollRef}
className="flex gap-4 sm:gap-5 overflow-x-auto scrollbar-hide pb-2 scroll-smooth"
style={{ scrollbarWidth: 'none', msOverflowStyle: 'none' }}
>
{series.map(s => (
<Link
key={s.asin}
href={`/series/${s.asin}${currentSeriesTitle ? `?from=${encodeURIComponent(currentSeriesTitle)}` : ''}`}
className="flex-shrink-0 w-20 sm:w-24 group/card outline-none focus-visible:ring-2 focus-visible:ring-emerald-500 rounded-xl"
>
{/* Cover */}
<div className={`relative w-20 ${squareCovers ? 'h-20 sm:w-24 sm:h-24' : 'h-[120px] sm:w-24 sm:h-36'} rounded-lg overflow-hidden shadow-md shadow-black/15 dark:shadow-black/30 group-hover/card:shadow-lg group-hover/card:scale-[1.04] group-hover/card:-translate-y-0.5 transition-all duration-300`}>
{s.coverArtUrl ? (
<Image
src={s.coverArtUrl}
alt=""
fill
className="object-cover"
sizes="96px"
/>
) : (
<div className="absolute inset-0 bg-gradient-to-br from-emerald-100 to-teal-200 dark:from-emerald-900 dark:to-teal-900 flex items-center justify-center">
<span className="text-lg font-bold text-emerald-400 dark:text-emerald-300">
{s.title.charAt(0).toUpperCase()}
</span>
</div>
)}
</div>
{/* Title */}
<p className="mt-2 text-xs sm:text-sm font-medium text-center text-gray-700 dark:text-gray-300 line-clamp-2 group-hover/card:text-emerald-600 dark:group-hover/card:text-emerald-400 transition-colors">
{s.title}
</p>
</Link>
))}
</div>
{/* Right arrow */}
{canScrollRight && (
<button
onClick={() => scroll('right')}
className="hidden md:flex absolute right-0 top-1/2 -translate-y-1/2 translate-x-3 z-10 w-10 h-10 bg-white dark:bg-gray-800 rounded-full shadow-lg items-center justify-center text-gray-600 dark:text-gray-300 hover:bg-gray-50 dark:hover:bg-gray-700 transition-all opacity-0 group-hover:opacity-100"
aria-label="Scroll right"
>
<svg className="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M9 5l7 7-7 7" />
</svg>
</button>
)}
{/* Fade edges */}
{canScrollLeft && (
<div className="hidden md:block absolute left-0 top-0 bottom-2 w-8 bg-gradient-to-r from-white dark:from-gray-900 to-transparent pointer-events-none z-[5]" />
)}
{canScrollRight && (
<div className="hidden md:block absolute right-0 top-0 bottom-2 w-8 bg-gradient-to-l from-white dark:from-gray-900 to-transparent pointer-events-none z-[5]" />
)}
</div>
</div>
);
}
export function SimilarSeriesSkeleton({ squareCovers = false }: { squareCovers?: boolean }) {
return (
<div className="space-y-3 animate-pulse">
<div className="flex items-center gap-3">
<div className="w-1 h-6 bg-gray-300 dark:bg-gray-600 rounded-full" />
<div className="h-7 w-40 bg-gray-200 dark:bg-gray-700 rounded-lg" />
</div>
<div className="flex gap-4 sm:gap-5 overflow-hidden">
{Array.from({ length: 8 }).map((_, i) => (
<div key={i} className="flex-shrink-0 w-20 sm:w-24" style={{ animationDelay: `${i * 50}ms` }}>
<div className={`w-20 ${squareCovers ? 'h-20 sm:w-24 sm:h-24' : 'h-[120px] sm:w-24 sm:h-36'} rounded-lg bg-gradient-to-br from-gray-200 to-gray-300 dark:from-gray-700 dark:to-gray-800 relative overflow-hidden`}>
<div className="absolute inset-0 -translate-x-full animate-[shimmer_2s_infinite] bg-gradient-to-r from-transparent via-white/20 to-transparent" />
</div>
<div className="mt-2 h-3 bg-gray-200 dark:bg-gray-700 rounded w-4/5 mx-auto" />
</div>
))}
</div>
</div>
);
}
+257
View File
@@ -0,0 +1,257 @@
/**
* Component: Centralized Language Configuration
* Documentation: documentation/integrations/audible.md
*
* Single source of truth for all language-specific configuration.
* To add a new language:
* 1. Add code to SupportedLanguage union
* 2. Add full LanguageConfig entry in LANGUAGE_CONFIGS
* 3. Map regions in REGION_LANGUAGE_MAP
* 4. Add region to AUDIBLE_REGIONS in audible.ts with language: 'xx'
*/
import type { AudibleRegion } from '../types/audible';
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
export type SupportedLanguage = 'en' | 'de' | 'es';
export interface ScrapingConfig {
/** Audible locale query-param value (e.g. 'english', 'deutsch') */
audibleLocaleParam: string;
/** Author label prefixes to strip (e.g. ['By:', 'Written by:']) */
authorPrefixes: string[];
/** Narrator label prefixes to strip */
narratorPrefixes: string[];
/** Length / duration labels used in Cheerio :contains() selectors */
lengthLabels: string[];
/** Language field labels */
languageLabels: string[];
/** Release date field labels */
releaseDateLabels: string[];
/** Series label prefixes used to find series links in search results */
seriesLabels: string[];
/** Accepted language values for filtering (lowercase) */
acceptedLanguageValues: string[];
/** Regex patterns that match hour portions in runtime strings */
runtimeHourPatterns: RegExp[];
/** Regex patterns that match minute portions in runtime strings */
runtimeMinutePatterns: RegExp[];
/** Regex patterns for extracting numeric rating */
ratingPatterns: RegExp[];
/** Regex patterns for extracting release date text */
releaseDatePatterns: RegExp[];
/** Promotional / non-description text patterns to exclude */
descriptionExcludePatterns: RegExp[];
/** Duration detection pattern for generic element scanning */
durationDetectionPattern: RegExp;
/** Rating text selector pattern (e.g. 'out of 5 stars') */
ratingTextSelector: string;
}
export interface LanguageConfig {
code: SupportedLanguage;
/** Anna's Archive language filter code */
annasArchiveLang: string;
/** EPUB language code */
epubCode: string;
/** Stop words for ranking algorithm (filtered from match scoring) */
stopWords: string[];
/** Character replacements applied before NFD normalization in ranking (e.g. ß→ss) */
characterReplacements: Record<string, string>;
/** All scraping-related config */
scraping: ScrapingConfig;
}
// ---------------------------------------------------------------------------
// Language Configurations
// ---------------------------------------------------------------------------
const ENGLISH_CONFIG: LanguageConfig = {
code: 'en',
annasArchiveLang: 'en',
epubCode: 'en',
stopWords: ['the', 'a', 'an', 'of', 'on', 'in', 'at', 'by', 'for'],
characterReplacements: {},
scraping: {
audibleLocaleParam: 'english',
authorPrefixes: ['By:', 'Written by:'],
narratorPrefixes: ['Narrated by:'],
lengthLabels: ['Length:'],
languageLabels: ['Language:'],
releaseDateLabels: ['Release date:'],
seriesLabels: ['Series:'],
acceptedLanguageValues: ['english'],
runtimeHourPatterns: [/(\d+)\s*hrs?/i, /(\d+)\s*hours?/i],
runtimeMinutePatterns: [/(\d+)\s*mins?/i, /(\d+)\s*minutes?/i],
ratingPatterns: [/(\d+\.?\d*)\s*out of/i],
releaseDatePatterns: [/Release date:\s*(.+)/i],
descriptionExcludePatterns: [
/\$\d+\.\d+/,
/cancel anytime/i,
/free trial/i,
/membership/i,
/subscribe/i,
/offer.*ends/i,
/^\s*by\s+[\w\s,]+$/i,
],
durationDetectionPattern: /\d+\s*(hr|hour|h)\s*\d*\s*(min|minute|m)?/i,
ratingTextSelector: 'out of 5 stars',
},
};
const GERMAN_CONFIG: LanguageConfig = {
code: 'de',
annasArchiveLang: 'de',
epubCode: 'de',
stopWords: ['der', 'die', 'das', 'ein', 'eine', 'und', 'von', 'zu', 'den', 'dem', 'des'],
characterReplacements: { '\u00df': 'ss' },
scraping: {
audibleLocaleParam: 'deutsch',
authorPrefixes: ['Von:', 'Geschrieben von:', 'Autor:'],
narratorPrefixes: ['Gesprochen von:', 'Sprecher:'],
lengthLabels: ['Spieldauer:', 'Dauer:', 'L\u00e4nge:'],
languageLabels: ['Sprache:'],
releaseDateLabels: ['Erscheinungsdatum:'],
seriesLabels: ['Serie:', 'Reihe:'],
acceptedLanguageValues: ['deutsch', 'german'],
runtimeHourPatterns: [/(\d+)\s*Std\.?/i, /(\d+)\s*Stunden?/i],
runtimeMinutePatterns: [/(\d+)\s*Min\.?/i, /(\d+)\s*Minuten?/i],
ratingPatterns: [/(\d+[.,]?\d*)\s*von\s*5/i],
releaseDatePatterns: [/Erscheinungsdatum:\s*(.+)/i],
descriptionExcludePatterns: [
/\$\d+\.\d+/,
/\d+,\d+\s*\u20ac/,
/jederzeit k\u00fcndbar/i,
/kostenlos testen/i,
/Mitgliedschaft/i,
/abonnieren/i,
/Angebot.*endet/i,
/^\s*von\s+[\w\s,]+$/i,
],
durationDetectionPattern: /\d+\s*(Std|Stunden?|h)\s*\.?\s*\d*\s*(Min|Minuten?|m)?/i,
ratingTextSelector: 'von 5 Sternen',
},
};
const SPANISH_CONFIG: LanguageConfig = {
code: 'es',
annasArchiveLang: 'es',
epubCode: 'es',
stopWords: ['el', 'la', 'los', 'las', 'un', 'una', 'de', 'del', 'en', 'y', 'por'],
characterReplacements: {},
scraping: {
audibleLocaleParam: 'espa\u00f1ol',
authorPrefixes: ['De:', 'Escrito por:', 'Autor:'],
narratorPrefixes: ['Narrado por:'],
lengthLabels: ['Duraci\u00f3n:'],
languageLabels: ['Idioma:'],
releaseDateLabels: ['Fecha de lanzamiento:'],
seriesLabels: ['Serie:'],
acceptedLanguageValues: ['espa\u00f1ol', 'spanish'],
runtimeHourPatterns: [/(\d+)\s*h\b/i, /(\d+)\s*horas?/i],
runtimeMinutePatterns: [/(\d+)\s*min/i, /(\d+)\s*minutos?/i],
ratingPatterns: [/(\d+[.,]?\d*)\s*de\s*5/i],
releaseDatePatterns: [/Fecha de lanzamiento:\s*(.+)/i],
descriptionExcludePatterns: [
/\$\d+\.\d+/,
/\d+,\d+\s*\u20ac/,
/cancela cuando quieras/i,
/prueba gratis/i,
/suscripci\u00f3n/i,
/suscr\u00edbete/i,
/oferta.*termina/i,
/^\s*de\s+[\w\s,]+$/i,
],
durationDetectionPattern: /\d+\s*(h|horas?)\s*\d*\s*(min|minutos?)?/i,
ratingTextSelector: 'de 5 estrellas',
},
};
// ---------------------------------------------------------------------------
// Lookup Maps
// ---------------------------------------------------------------------------
export const LANGUAGE_CONFIGS: Record<SupportedLanguage, LanguageConfig> = {
en: ENGLISH_CONFIG,
de: GERMAN_CONFIG,
es: SPANISH_CONFIG,
};
/**
* Maps Audible region codes to language codes.
* All English-speaking regions map to 'en'.
*/
export const REGION_LANGUAGE_MAP: Record<AudibleRegion, SupportedLanguage> = {
us: 'en',
ca: 'en',
uk: 'en',
au: 'en',
in: 'en',
de: 'de',
es: 'es',
};
// ---------------------------------------------------------------------------
// Helper Functions
// ---------------------------------------------------------------------------
/**
* Get the full language configuration for an Audible region.
*/
export function getLanguageForRegion(region: AudibleRegion): LanguageConfig {
const langCode = REGION_LANGUAGE_MAP[region];
return LANGUAGE_CONFIGS[langCode];
}
/**
* Strip any matching prefixes from text (case-insensitive).
* Returns the text with the first matching prefix removed, trimmed.
*
* Example: stripPrefixes('By: Author Name', ['By:', 'Written by:']) => 'Author Name'
*/
export function stripPrefixes(text: string, prefixes: string[]): string {
const trimmed = text.trim();
for (const prefix of prefixes) {
if (trimmed.toLowerCase().startsWith(prefix.toLowerCase())) {
return trimmed.slice(prefix.length).trim();
}
}
return trimmed;
}
/**
* Build a Cheerio selector that matches any of the given labels using :contains().
* Returns a comma-separated selector string.
*
* Example: buildContainsSelector('span', ['Length:', 'Dauer:'])
* => 'span:contains("Length:"), span:contains("Dauer:")'
*/
export function buildContainsSelector(element: string, labels: string[]): string {
return labels.map(label => `${element}:contains("${label}")`).join(', ');
}
/**
* Extract a value from text by trying multiple label patterns.
* Returns the captured group from the first matching pattern, or null.
*/
export function extractByPatterns(text: string, patterns: RegExp[]): string | null {
for (const pattern of patterns) {
const match = text.match(pattern);
if (match?.[1]) {
return match[1].trim();
}
}
return null;
}
/**
* Check if a language value matches the accepted values for a language config.
* Comparison is case-insensitive.
*/
export function isAcceptedLanguage(languageValue: string, config: LanguageConfig): boolean {
const normalized = languageValue.toLowerCase().trim();
return config.scraping.acceptedLanguageValues.includes(normalized);
}
+28
View File
@@ -5,6 +5,29 @@
import { PrismaClient } from '@/generated/prisma/client';
/**
* Append connection pool parameters to DATABASE_URL if not already present.
* - connection_limit=20: up from default 9, fits 22 max workers + API routes
* - pool_timeout=30: up from default 10s, gives queued requests time
*/
function getPooledDatabaseUrl(): string {
const baseUrl = process.env.DATABASE_URL || '';
if (!baseUrl) return baseUrl;
const separator = baseUrl.includes('?') ? '&' : '?';
const params: string[] = [];
if (!baseUrl.includes('connection_limit')) {
params.push('connection_limit=20');
}
if (!baseUrl.includes('pool_timeout')) {
params.push('pool_timeout=30');
}
if (params.length === 0) return baseUrl;
return `${baseUrl}${separator}${params.join('&')}`;
}
// Prevent multiple instances of Prisma Client in development
const globalForPrisma = globalThis as unknown as {
prisma: PrismaClient | undefined;
@@ -14,6 +37,11 @@ export const prisma =
globalForPrisma.prisma ??
new PrismaClient({
log: process.env.NODE_ENV === 'development' ? ['query', 'error', 'warn'] : ['error'],
datasources: {
db: {
url: getPooledDatabaseUrl(),
},
},
});
if (process.env.NODE_ENV !== 'production') globalForPrisma.prisma = prisma;
+3
View File
@@ -20,6 +20,9 @@ export interface Audiobook {
releaseDate?: string;
rating?: number;
genres?: string[];
series?: string; // Series name (e.g., "A Song of Ice and Fire")
seriesPart?: string; // Position in series (e.g., "1", "1.5")
seriesAsin?: string; // Audible ASIN for the series (links to /series/{asin})
isAvailable?: boolean; // Set by real-time matching against plex_library
plexGuid?: string | null;
dbId?: string | null;
+75
View File
@@ -0,0 +1,75 @@
/**
* Component: Series Fetching Hooks
* Documentation: documentation/frontend/components.md
*/
'use client';
import useSWR from 'swr';
import { authenticatedFetcher } from '@/lib/utils/api';
import { Audiobook } from './useAudiobooks';
export interface SeriesSummary {
asin: string;
title: string;
bookCount: number;
rating?: number;
ratingCount?: number;
tags: string[];
coverArtUrl?: string;
audibleUrl: string;
}
export interface SimilarSeries {
asin: string;
title: string;
bookCount?: number;
coverArtUrl?: string;
}
export interface SeriesDetail {
asin: string;
title: string;
bookCount: number;
rating?: number;
ratingCount?: number;
description?: string;
tags: string[];
books: Audiobook[];
similarSeries: SimilarSeries[];
audibleUrl: string;
}
export function useSeriesSearch(query: string) {
const shouldFetch = query && query.length > 0;
const endpoint = shouldFetch
? `/api/series/search?q=${encodeURIComponent(query)}`
: null;
const { data, error, isLoading } = useSWR(endpoint, authenticatedFetcher, {
revalidateOnFocus: false,
dedupingInterval: 30000,
});
return {
series: (data?.series || []) as SeriesSummary[],
query: data?.query || '',
isLoading: shouldFetch && isLoading,
error,
};
}
export function useSeriesDetail(asin: string | null) {
const endpoint = asin ? `/api/series/${asin}` : null;
const { data, error, isLoading } = useSWR(endpoint, authenticatedFetcher, {
revalidateOnFocus: false,
dedupingInterval: 300000, // Cache for 5 minutes
});
return {
series: (data?.series || null) as SeriesDetail | null,
isLoading,
error,
};
}
+515
View File
@@ -0,0 +1,515 @@
/**
* Component: Audible Series Scraping
* Documentation: documentation/integrations/audible.md
*
* Standalone series scraping module. Uses the AudibleService fetch wrapper
* for HTTP requests and Cheerio for HTML parsing.
* Kept separate from audible.service.ts to avoid bloating the main service.
*/
import * as cheerio from 'cheerio';
import { getAudibleService, AudibleAudiobook } from './audible.service';
import { AUDIBLE_REGIONS } from '../types/audible';
import {
getLanguageForRegion,
buildContainsSelector,
stripPrefixes,
} from '../constants/language-config';
import { RMABLogger } from '../utils/logger';
import { randomDelay } from '../utils/scrape-resilience';
const logger = RMABLogger.create('Audible.Series');
const AUDIBLE_PAGE_SIZE = 50;
const MAX_SERIES_RESULTS = 15;
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
export interface SeriesSummary {
asin: string;
title: string;
bookCount: number;
rating?: number;
ratingCount?: number;
tags: string[];
coverArtUrl?: string;
audibleUrl: string;
}
export interface SimilarSeries {
asin: string;
title: string;
bookCount?: number;
coverArtUrl?: string;
}
export interface SeriesDetail {
asin: string;
title: string;
bookCount: number;
rating?: number;
ratingCount?: number;
description?: string;
tags: string[];
books: AudibleAudiobook[];
similarSeries: SimilarSeries[];
audibleUrl: string;
}
// ---------------------------------------------------------------------------
// Search: extract series links from Audible search results
// ---------------------------------------------------------------------------
/**
* Search for series by scraping Audible search results and extracting
* series links. De-duplicates by ASIN, then scrapes each unique series
* page in parallel (capped at MAX_SERIES_RESULTS).
*/
export async function searchForSeries(query: string): Promise<SeriesSummary[]> {
const service = getAudibleService();
const region = service.getRegion();
const baseUrl = service.getBaseUrl();
const langConfig = getLanguageForRegion(region);
const seriesLabels = langConfig.scraping.seriesLabels;
logger.info(`Searching series for "${query}" (region: ${region})`);
// Step 1: Fetch search results page
let $: cheerio.CheerioAPI;
try {
const { data: response } = await service.fetch('/search', {
params: {
ipRedirectOverride: 'true',
keywords: query,
pageSize: AUDIBLE_PAGE_SIZE,
},
});
$ = cheerio.load(response.data);
} catch (error) {
logger.error('Series search fetch failed', {
error: error instanceof Error ? error.message : String(error),
});
return [];
}
// Step 2: Extract unique series ASINs from search results
// Series links appear inside spans containing locale-specific "Series:" text
const seriesMap = new Map<string, { title: string; coverArtUrl?: string }>();
$('.s-result-item, .productListItem').each((_index, element) => {
if (seriesMap.size >= MAX_SERIES_RESULTS) return false;
const $el = $(element);
// Find the span containing a series label (e.g. "Series:")
const seriesSelector = buildContainsSelector('span', seriesLabels);
const seriesContainer = $el.find(seriesSelector).first();
if (seriesContainer.length === 0) return;
// Look for series link within or near the series label container
// The series link is a child or sibling: <a href="/series/Name/B006K1QER6">
const parentEl = seriesContainer.parent();
const seriesLink = parentEl.find('a[href*="/series/"]').first();
if (seriesLink.length === 0) return;
const href = seriesLink.attr('href') || '';
const asinMatch = href.match(/\/series\/[^/]*\/([A-Z0-9]{10})/);
if (!asinMatch) return;
const asin = asinMatch[1];
if (seriesMap.has(asin)) return;
const title = seriesLink.text().trim();
if (!title) return;
// Use the first book's cover as representative image
const coverArtUrl = $el.find('img').first().attr('src')?.replace(/\._.*_\./, '._SL500_.') || undefined;
seriesMap.set(asin, { title, coverArtUrl });
});
if (seriesMap.size === 0) {
logger.info(`No series found for "${query}"`);
return [];
}
logger.info(`Found ${seriesMap.size} unique series, scraping detail pages...`);
// Step 3: Scrape each series page in parallel (with rate limiting)
const entries = Array.from(seriesMap.entries());
const BATCH_SIZE = 5;
const results: SeriesSummary[] = [];
for (let i = 0; i < entries.length; i += BATCH_SIZE) {
const batch = entries.slice(i, i + BATCH_SIZE);
const batchResults = await Promise.all(
batch.map(async ([asin, meta]) => {
try {
const detail = await scrapeSeriesPageSummary(asin);
if (!detail) return null;
return {
...detail,
coverArtUrl: detail.coverArtUrl || meta.coverArtUrl,
audibleUrl: `${baseUrl}/series/${asin}`,
} as SeriesSummary;
} catch (error) {
logger.warn(`Failed to scrape series ${asin}`, {
error: error instanceof Error ? error.message : String(error),
});
// Return a minimal result from search data
return {
asin,
title: meta.title,
bookCount: 0,
tags: [],
coverArtUrl: meta.coverArtUrl,
audibleUrl: `${baseUrl}/series/${asin}`,
} as SeriesSummary;
}
})
);
results.push(...batchResults.filter((r): r is SeriesSummary => r !== null));
// Rate limit between batches
if (i + BATCH_SIZE < entries.length) {
await new Promise(resolve => setTimeout(resolve, randomDelay(1500, 3000)));
}
}
logger.info(`Series search complete: "${query}" -> ${results.length} results`);
return results;
}
// ---------------------------------------------------------------------------
// Series page scraping (summary - for search results)
// ---------------------------------------------------------------------------
/**
* Scrape a series page for summary data (title, book count, rating, tags).
* Used during search to enrich each series result.
*/
async function scrapeSeriesPageSummary(asin: string): Promise<Omit<SeriesSummary, 'audibleUrl'> | null> {
const service = getAudibleService();
try {
const { data: response } = await service.fetch(`/series/${asin}`, {
params: { ipRedirectOverride: 'true' },
});
const $ = cheerio.load(response.data);
return parseSeriesPageSummary($, asin);
} catch (error) {
logger.warn(`Failed to fetch series page ${asin}`, {
error: error instanceof Error ? error.message : String(error),
});
return null;
}
}
/**
* Parse summary fields from a series page's Cheerio document.
*/
function parseSeriesPageSummary(
$: cheerio.CheerioAPI,
asin: string
): Omit<SeriesSummary, 'audibleUrl'> {
// Title - from h1
const title = $('h1').first().text().trim() || '';
// Book count - multiple strategies, most specific first
let bookCount = 0;
// Primary: adbl-metadata[slot="child-count"] in the page header (NOT inside carousels)
// Filter out carousel items by excluding those inside adbl-product-carousel
$('adbl-metadata[slot="child-count"]').each((_i, el) => {
if (bookCount > 0) return false;
const $el = $(el);
// Skip if inside a carousel (those are similar-series counts)
if ($el.closest('adbl-product-carousel').length > 0) return;
const text = $el.text().trim();
const match = text.match(/(\d+)/);
if (match) bookCount = parseInt(match[1]);
});
// Secondary: text matching in spans/headings for "X books/titles/Titel/libros/Bucher"
if (bookCount === 0) {
const countText = $('span:contains("book"), span:contains("title"), span:contains("Titel"), span:contains("libro"), span:contains("Buch"), span:contains("B\u00fccher")')
.text().trim();
const countMatch = countText.match(/(\d+)\s*(books?|titles?|Titel|libros?|B(?:uch|\u00fccher))/i);
if (countMatch) {
bookCount = parseInt(countMatch[1]);
}
}
// Fallback: count product items on the page
if (bookCount === 0) {
bookCount = $('.productListItem, .bc-list-item[data-asin]').length;
}
// Rating
const { rating, ratingCount } = parseSeriesRating($);
// Tags/genres: primary from adbl-chip web components, fallback to legacy links
const tags: string[] = [];
const addTag = (text: string) => {
const tag = text.trim();
if (tag && tag.length >= 2 && tag.length <= 50 && !tags.includes(tag)) {
tags.push(tag);
}
};
// Primary: adbl-chip.related-tag elements (modern Audible layout)
$('adbl-chip.related-tag').each((_i, el) => {
addTag($(el).text());
});
// Fallback: legacy category and tag links
if (tags.length === 0) {
$('a[href*="/cat/"], a[href*="/tag/"]').each((_i, el) => {
addTag($(el).text());
});
}
// Cover art from first book image
const coverArtUrl = $('.productListItem img, .bc-list-item img').first()
.attr('src')?.replace(/\._.*_\./, '._SL500_.') || undefined;
return { asin, title, bookCount, rating, ratingCount, tags: tags.slice(0, 5), coverArtUrl };
}
// ---------------------------------------------------------------------------
// Series page scraping (full detail)
// ---------------------------------------------------------------------------
/**
* Scrape a series page for full detail data including books and similar series.
* Used by the detail API endpoint.
*/
export async function scrapeSeriesPage(asin: string): Promise<SeriesDetail | null> {
const service = getAudibleService();
const region = service.getRegion();
const baseUrl = service.getBaseUrl();
const langConfig = getLanguageForRegion(region);
logger.info(`Scraping series detail page: ${asin}`);
try {
const { data: response } = await service.fetch(`/series/${asin}`, {
params: { ipRedirectOverride: 'true', pageSize: AUDIBLE_PAGE_SIZE },
});
const $ = cheerio.load(response.data);
// Parse summary fields
const summary = parseSeriesPageSummary($, asin);
// Description
const description = $('.bc-expander-content').first().text().trim() ||
$('[class*="productPublisherSummary"]').first().text().trim() ||
undefined;
// Parse all books from the series page
const books = parseSeriesBooks($, langConfig.scraping.authorPrefixes, langConfig.scraping.narratorPrefixes);
// Use actual book count if we got more from scraping
const bookCount = Math.max(summary.bookCount, books.length);
// Parse similar series ("Listeners also enjoyed" or similar section)
const similarSeries = parseSimilarSeries($);
logger.info(`Series detail complete: "${summary.title}" (${books.length} books, ${similarSeries.length} similar)`);
return {
asin,
title: summary.title,
bookCount,
rating: summary.rating,
ratingCount: summary.ratingCount,
description,
tags: summary.tags,
books,
similarSeries,
audibleUrl: `${baseUrl}/series/${asin}`,
};
} catch (error) {
logger.error(`Failed to scrape series detail ${asin}`, {
error: error instanceof Error ? error.message : String(error),
});
return null;
}
}
// ---------------------------------------------------------------------------
// Parsing helpers
// ---------------------------------------------------------------------------
/**
* Extract rating and rating count from a series page.
*
* Real HTML uses:
* <div aria-label="4.5 out of 5 stars" class="bc-review-stars ...">
* <span class="series-rating bc-color-secondary">8,704 ratings</span>
*/
function parseSeriesRating($: cheerio.CheerioAPI): { rating?: number; ratingCount?: number } {
let rating: number | undefined;
let ratingCount: number | undefined;
// Primary: aria-label on div.bc-review-stars (e.g. "4.5 out of 5 stars")
const starsDiv = $('div.bc-review-stars');
let ariaLabel = starsDiv.attr('aria-label') || '';
// Fallback: any element with aria-label containing rating pattern
if (!ariaLabel) {
const fallbackEl = $('[aria-label*="out of"], [aria-label*="von 5"], [aria-label*="de 5"]').first();
ariaLabel = fallbackEl.attr('aria-label') || '';
}
// Extract numeric rating from aria-label (handles "4.5 out of 5", "4,5 von 5", "4,5 de 5")
const ratingMatch = ariaLabel.match(/(\d+[.,]?\d*)\s*(?:out of|von|de)\s*5/i);
if (ratingMatch) {
rating = parseFloat(ratingMatch[1].replace(',', '.'));
}
// Rating count from span.series-rating (e.g. "8,704 ratings")
const seriesRatingSpan = $('span.series-rating').first();
let countText = seriesRatingSpan.text().trim();
// Fallback: look in broader context for rating count text
if (!countText) {
const fallbackContainer = $('[class*="rating"], .ratingsLabel').first();
countText = fallbackContainer.text().trim();
}
const countMatch = countText.match(/([\d,.]+)\s*(?:ratings?|Bewertungen?|calificaciones?)/i);
if (countMatch) {
ratingCount = parseInt(countMatch[1].replace(/[.,]/g, ''));
}
return { rating, ratingCount };
}
/**
* Parse all books from a series page's product list items.
*/
function parseSeriesBooks(
$: cheerio.CheerioAPI,
authorPrefixes: string[],
narratorPrefixes: string[]
): AudibleAudiobook[] {
const books: AudibleAudiobook[] = [];
const seenAsins = new Set<string>();
$('.productListItem, .bc-list-item').each((_index, element) => {
const $el = $(element);
// Extract ASIN
const bookAsin = $el.attr('data-asin') ||
$el.find('li').attr('data-asin') ||
$el.find('a[href*="/pd/"]').attr('href')?.match(/\/pd\/[^/]+\/([A-Z0-9]{10})/)?.[1] ||
$el.find('a[href*="/ac/"]').attr('href')?.match(/\/ac\/[^/]+\/([A-Z0-9]{10})/)?.[1] ||
$el.find('a').attr('href')?.match(/\/(?:pd|ac)\/[^/]+\/([A-Z0-9]{10})/)?.[1] || '';
if (!bookAsin || seenAsins.has(bookAsin)) return;
seenAsins.add(bookAsin);
// Title
const title = $el.find('h2').first().text().trim() ||
$el.find('h3 a').first().text().trim() ||
$el.find('.bc-heading a').first().text().trim() ||
'';
if (!title) return;
// Author
const authorLink = $el.find('a[href*="/author/"]').first();
const authorText = authorLink.text().trim() ||
$el.find('.authorLabel').text().trim() ||
'';
const authorHref = authorLink.attr('href') || '';
const authorAsinMatch = authorHref.match(/\/author\/[^/]+\/([A-Z0-9]{10})/);
// Narrator
const narratorText = $el.find('a[href*="searchNarrator="]').first().text().trim() ||
$el.find('.narratorLabel').text().trim() ||
'';
// Cover art
const coverArtUrl = $el.find('img').first().attr('src')?.replace(/\._.*_\./, '._SL500_.') || '';
// Rating
const ratingText = $el.find('.ratingsLabel').text().trim() ||
$el.find('.a-icon-star span').first().text().trim();
const ratingMatch = ratingText ? ratingText.match(/(\d+[.,]?\d*)/) : null;
const rating = ratingMatch ? parseFloat(ratingMatch[1].replace(',', '.')) : undefined;
books.push({
asin: bookAsin,
title,
author: stripPrefixes(authorText, authorPrefixes),
authorAsin: authorAsinMatch?.[1] || undefined,
narrator: stripPrefixes(narratorText, narratorPrefixes),
coverArtUrl,
rating,
});
});
return books;
}
/**
* Parse similar series from the "Listeners also enjoyed" carousel.
*
* Real HTML uses web components:
* <adbl-product-carousel id="SeriestoSeries">
* <adbl-product-grid-item>
* <div class="adbl-impression-emitted" data-asin="B0CGS1LPWJ">
* <adbl-metadata slot="title"><a>Hockey Guys</a></adbl-metadata>
* <adbl-metadata slot="child-count">3 titles</adbl-metadata>
* </adbl-product-grid-item>
*/
function parseSimilarSeries($: cheerio.CheerioAPI): SimilarSeries[] {
const similar: SimilarSeries[] = [];
const seenAsins = new Set<string>();
// Scope to the SeriestoSeries carousel to avoid picking up other series links
const carousel = $('adbl-product-carousel#SeriestoSeries');
if (carousel.length === 0) return similar;
carousel.find('adbl-product-grid-item').each((_i, el) => {
if (similar.length >= 15) return false;
const $el = $(el);
// Extract ASIN: prefer data-asin on impression div, fallback to series href
let asin = $el.find('.adbl-impression-emitted, .adbl-asin-impression').first().attr('data-asin') || '';
if (!asin) {
const seriesHref = $el.find('a[href*="/series/"]').first().attr('href') || '';
const hrefMatch = seriesHref.match(/\/series\/[^/]*\/([A-Z0-9]{10})/);
if (hrefMatch) asin = hrefMatch[1];
}
if (!asin || !/^[A-Z0-9]{10}$/.test(asin)) return;
if (seenAsins.has(asin)) return;
seenAsins.add(asin);
// Title from metadata slot
const title = $el.find('adbl-metadata[slot="title"] a').first().text().trim() ||
$el.find('adbl-metadata[slot="title"]').first().text().trim() || '';
if (!title || title.length > 200) return;
// Book count from child-count slot (e.g. "3 titles")
const countText = $el.find('adbl-metadata[slot="child-count"]').first().text().trim();
const countMatch = countText.match(/(\d+)/);
const bookCount = countMatch ? parseInt(countMatch[1]) : undefined;
// Cover image from adbl-collection-image
const coverArtUrl = $el.find('adbl-collection-image img').first().attr('src')?.replace(/\._.*_\./, '._SL500_.') ||
$el.find('img').first().attr('src')?.replace(/\._.*_\./, '._SL500_.') ||
undefined;
similar.push({ asin, title, bookCount, coverArtUrl });
});
return similar;
}
+119 -50
View File
@@ -8,6 +8,14 @@ import * as cheerio from 'cheerio';
import { RMABLogger } from '../utils/logger';
import { getConfigService } from '../services/config.service';
import { AudibleRegion, AUDIBLE_REGIONS, DEFAULT_AUDIBLE_REGION } from '../types/audible';
import {
getLanguageForRegion,
stripPrefixes,
buildContainsSelector,
extractByPatterns,
isAcceptedLanguage,
type LanguageConfig,
} from '../constants/language-config';
import {
pickUserAgent,
getBrowserHeaders,
@@ -40,6 +48,7 @@ export interface AudibleAudiobook {
genres?: string[];
series?: string;
seriesPart?: string;
seriesAsin?: string;
}
export interface AudibleSearchResult {
@@ -69,6 +78,29 @@ export class AudibleService {
return this.baseUrl;
}
/**
* Get the current Audible region code
*/
public getRegion(): AudibleRegion {
return this.region;
}
/**
* Public fetch wrapper for external scraping modules (e.g. audible-series.ts).
* Ensures the service is initialized and delegates to fetchWithRetry.
*/
public async fetch(url: string, config: any = {}): Promise<{ data: any; meta: FetchResultMeta }> {
await this.initialize();
return this.fetchWithRetry(url, config);
}
/**
* Get the language config for the current region
*/
private getLangConfig(): LanguageConfig {
return getLanguageForRegion(this.region);
}
/**
* Force re-initialization (used when region config changes)
*/
@@ -106,6 +138,9 @@ export class AudibleService {
logger.info(`Initializing Audible service with region: ${this.region} (${this.baseUrl})`);
// Get language config for the region
const langConfig = getLanguageForRegion(this.region);
// Create axios client with region-specific base URL and realistic browser headers
this.client = axios.create({
baseURL: this.baseUrl,
@@ -113,7 +148,7 @@ export class AudibleService {
headers: getBrowserHeaders(this.sessionUserAgent),
params: {
ipRedirectOverride: 'true', // Prevent IP-based region redirects
language: 'english', // Force English locale (prevents IP-based language serving for non-English IPs)
language: langConfig.scraping.audibleLocaleParam, // Force locale (prevents IP-based language serving)
},
});
@@ -125,13 +160,16 @@ export class AudibleService {
this.baseUrl = AUDIBLE_REGIONS[this.region].baseUrl;
this.sessionUserAgent = pickUserAgent();
this.pacer.reset();
const fallbackLangConfig = getLanguageForRegion(this.region);
this.client = axios.create({
baseURL: this.baseUrl,
timeout: 15000,
headers: getBrowserHeaders(this.sessionUserAgent),
params: {
ipRedirectOverride: 'true',
language: 'english',
language: fallbackLangConfig.scraping.audibleLocaleParam,
},
});
this.initialized = true;
@@ -289,12 +327,14 @@ export class AudibleService {
const ratingText = $el.find('.ratingsLabel').text().trim();
const rating = ratingText ? parseFloat(ratingText.split(' ')[0]) : undefined;
const langConfig = this.getLangConfig();
audiobooks.push({
asin,
title,
author: authorText.replace('By:', '').replace('Written by:', '').trim(),
author: stripPrefixes(authorText, langConfig.scraping.authorPrefixes),
authorAsin: authorAsinMatch?.[1] || undefined,
narrator: narratorText.replace('Narrated by:', '').trim(),
narrator: stripPrefixes(narratorText, langConfig.scraping.narratorPrefixes),
coverArtUrl: coverArtUrl.replace(/\._.*_\./, '._SL500_.'),
rating,
});
@@ -391,12 +431,14 @@ export class AudibleService {
const ratingText = $el.find('.ratingsLabel').text().trim();
const rating = ratingText ? parseFloat(ratingText.split(' ')[0]) : undefined;
const langConfig = this.getLangConfig();
audiobooks.push({
asin,
title,
author: authorText.replace('By:', '').replace('Written by:', '').trim(),
author: stripPrefixes(authorText, langConfig.scraping.authorPrefixes),
authorAsin: authorAsinMatch?.[1] || undefined,
narrator: narratorText.replace('Narrated by:', '').trim(),
narrator: stripPrefixes(narratorText, langConfig.scraping.narratorPrefixes),
coverArtUrl: coverArtUrl.replace(/\._.*_\./, '._SL500_.'),
rating,
});
@@ -487,9 +529,11 @@ export class AudibleService {
const coverArtUrl = $el.find('img').attr('src') || '';
const langConfig = this.getLangConfig();
// Extract runtime/duration
const runtimeText = $el.find('.runtimeLabel').text().trim() ||
$el.find('span:contains("Length:")').text().trim();
$el.find(buildContainsSelector('span', langConfig.scraping.lengthLabels)).text().trim();
const durationMinutes = this.parseRuntime(runtimeText);
// Extract rating
@@ -500,9 +544,9 @@ export class AudibleService {
audiobooks.push({
asin,
title,
author: authorText.replace('By:', '').replace('Written by:', '').trim(),
author: stripPrefixes(authorText, langConfig.scraping.authorPrefixes),
authorAsin: authorAsinMatch?.[1] || undefined,
narrator: narratorText.replace('Narrated by:', '').trim(),
narrator: stripPrefixes(narratorText, langConfig.scraping.narratorPrefixes),
coverArtUrl: coverArtUrl.replace(/\._.*_\./, '._SL500_.'),
durationMinutes,
rating,
@@ -565,13 +609,15 @@ export class AudibleService {
$('.s-result-item, .productListItem').each((_index, element) => {
const $el = $(element);
// --- Language filter: require explicit "English" ---
const langText = $el.find('span:contains("Language:")').text().trim() ||
// --- Language filter: require matching language for region ---
const langConfig = this.getLangConfig();
const langText = $el.find(buildContainsSelector('span', langConfig.scraping.languageLabels)).text().trim() ||
$el.find('.languageLabel').text().trim();
// Extract language value (e.g. "Language: English" "English")
const langMatch = langText.match(/Language:\s*(.+)/i);
// Extract language value (e.g. "Language: English" -> "English", "Sprache: Deutsch" -> "Deutsch")
const langLabelPattern = new RegExp(`(?:${langConfig.scraping.languageLabels.map(l => l.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')).join('|')})\\s*(.+)`, 'i');
const langMatch = langText.match(langLabelPattern);
const language = langMatch?.[1]?.trim();
if (!language || language.toLowerCase() !== 'english') return;
if (!language || !isAcceptedLanguage(language, langConfig)) return;
// --- Author ASIN filter: verify target ASIN in author links ---
const authorLinks = $el.find('a[href*="/author/"]');
@@ -609,7 +655,7 @@ export class AudibleService {
const coverArtUrl = $el.find('img').attr('src') || '';
const runtimeText = $el.find('.runtimeLabel').text().trim() ||
$el.find('span:contains("Length:")').text().trim();
$el.find(buildContainsSelector('span', langConfig.scraping.lengthLabels)).text().trim();
const durationMinutes = this.parseRuntime(runtimeText);
const ratingText = $el.find('.ratingsLabel').text().trim() ||
@@ -619,9 +665,9 @@ export class AudibleService {
allBooks.push({
asin: bookAsin,
title,
author: authorText.replace('By:', '').replace('Written by:', '').trim(),
author: stripPrefixes(authorText, langConfig.scraping.authorPrefixes),
authorAsin,
narrator: narratorText.replace('Narrated by:', '').trim(),
narrator: stripPrefixes(narratorText, langConfig.scraping.narratorPrefixes),
coverArtUrl: coverArtUrl.replace(/\._.*_\./, '._SL500_.'),
durationMinutes,
rating,
@@ -720,6 +766,7 @@ export class AudibleService {
genres: data.genres?.map((g: any) => typeof g === 'string' ? g : g.name).slice(0, 5) || undefined,
series: data.seriesPrimary?.name || undefined,
seriesPart: data.seriesPrimary?.position || undefined,
seriesAsin: data.seriesPrimary?.asin || undefined,
};
// Ensure cover art URL is high quality
@@ -736,7 +783,8 @@ export class AudibleService {
rating: result.rating,
genreCount: result.genres?.length || 0,
series: result.series,
seriesPart: result.seriesPart
seriesPart: result.seriesPart,
seriesAsin: result.seriesAsin
});
return result;
@@ -867,7 +915,8 @@ export class AudibleService {
result.author = [...new Set(authors)].slice(0, 3).join(', ');
}
result.author = result.author.replace(/^By:\s*/i, '').replace(/^Written by:\s*/i, '').trim();
const authorLangConfig = this.getLangConfig();
result.author = stripPrefixes(result.author, authorLangConfig.scraping.authorPrefixes);
logger.info(` Author from HTML: "${result.author}"`);
}
@@ -911,22 +960,16 @@ export class AudibleService {
}
if (result.narrator) {
result.narrator = result.narrator.replace(/^Narrated by:\s*/i, '').trim();
const detailLangConfig = this.getLangConfig();
result.narrator = stripPrefixes(result.narrator, detailLangConfig.scraping.narratorPrefixes);
}
logger.info(` Narrator from HTML: "${result.narrator || ''}"`);
}
// Description - try multiple approaches with strict filtering
if (!result.description) {
const excludePatterns = [
/\$\d+\.\d+/, // Price patterns
/cancel anytime/i,
/free trial/i,
/membership/i,
/subscribe/i,
/offer.*ends/i,
/^\s*by\s+[\w\s,]+$/i, // Just author names
];
const descLangConfig = this.getLangConfig();
const excludePatterns = descLangConfig.scraping.descriptionExcludePatterns;
const isValidDescription = (text: string): boolean => {
if (!text || text.length < 50 || text.length > 5000) return false;
@@ -982,18 +1025,20 @@ export class AudibleService {
// Runtime/Duration - try multiple approaches
if (!result.durationMinutes) {
const rtLangConfig = this.getLangConfig();
// Look for runtime text in various places
const runtimeText =
$('li.runtimeLabel span').text().trim() ||
$('.runtimeLabel').text().trim() ||
$('span:contains("Length:")').parent().text().trim() ||
$('li:contains("Length:")').text().trim() ||
$(buildContainsSelector('span', rtLangConfig.scraping.lengthLabels)).parent().text().trim() ||
$(buildContainsSelector('li', rtLangConfig.scraping.lengthLabels)).text().trim() ||
(() => {
// Look for any text matching duration pattern
let found = '';
$('li, span, div').each((_, elem) => {
const text = $(elem).text().trim();
if (text.match(/\d+\s*(hr|hour|h)\s*\d*\s*(min|minute|m)?/i) && text.length < 100) {
if (text.match(rtLangConfig.scraping.durationDetectionPattern) && text.length < 100) {
found = text;
return false; // break
}
@@ -1007,41 +1052,55 @@ export class AudibleService {
// Rating - try multiple approaches
if (!result.rating) {
const ratingLangConfig = this.getLangConfig();
const ratingText =
$('.ratingsLabel').text().trim() ||
$('[class*="rating"]').first().text().trim() ||
$('span:contains("out of 5 stars")').parent().text().trim() ||
$(`span:contains("${ratingLangConfig.scraping.ratingTextSelector}")`).parent().text().trim() ||
(() => {
// Look for rating pattern
// Look for rating pattern using language-specific patterns
let found = '';
$('span, div').each((_, elem) => {
const text = $(elem).text().trim();
if (text.match(/\d+\.?\d*\s*out of\s*5/i) && text.length < 50) {
if (text.length < 50) {
for (const pattern of ratingLangConfig.scraping.ratingPatterns) {
if (pattern.test(text)) {
found = text;
return false;
}
}
}
});
return found;
})();
if (ratingText) {
const ratingMatch = ratingText.match(/(\d+\.?\d*)\s*out of/i);
result.rating = ratingMatch ? parseFloat(ratingMatch[1]) : undefined;
let ratingValue: number | undefined;
for (const pattern of ratingLangConfig.scraping.ratingPatterns) {
const ratingMatch = ratingText.match(pattern);
if (ratingMatch) {
// Handle comma as decimal separator (e.g. "4,5" in German/Spanish)
ratingValue = parseFloat(ratingMatch[1].replace(',', '.'));
break;
}
}
result.rating = ratingValue;
}
logger.info(` Rating from "${ratingText}": ${result.rating}`);
}
// Release date - try multiple selectors
if (!result.releaseDate) {
const rdLangConfig = this.getLangConfig();
const releaseDateText =
$('li:contains("Release date:")').text().trim() ||
$('span:contains("Release date:")').parent().text().trim() ||
$(buildContainsSelector('li', rdLangConfig.scraping.releaseDateLabels)).text().trim() ||
$(buildContainsSelector('span', rdLangConfig.scraping.releaseDateLabels)).parent().text().trim() ||
$('[class*="release"]').text().trim();
const dateMatch = releaseDateText.match(/Release date:\s*(.+)/i) ||
releaseDateText.match(/(\w+ \d{1,2},? \d{4})/);
const dateMatch = extractByPatterns(releaseDateText, rdLangConfig.scraping.releaseDatePatterns) ||
releaseDateText.match(/(\w+ \d{1,2},? \d{4})/)?.[1];
if (dateMatch) {
result.releaseDate = dateMatch[1].trim();
result.releaseDate = dateMatch.trim();
}
logger.info(` Release date from "${releaseDateText}": ${result.releaseDate}`);
}
@@ -1078,20 +1137,30 @@ export class AudibleService {
}
/**
* Parse runtime text to minutes
* Parse runtime text to minutes using language-specific patterns
*/
private parseRuntime(runtimeText: string): number | undefined {
if (!runtimeText) return undefined;
const hoursMatch = runtimeText.match(/(\d+)\s*hrs?/i);
const minutesMatch = runtimeText.match(/(\d+)\s*mins?/i);
const langConfig = this.getLangConfig();
let totalMinutes = 0;
if (hoursMatch) {
totalMinutes += parseInt(hoursMatch[1]) * 60;
// Try each hour pattern until one matches
for (const pattern of langConfig.scraping.runtimeHourPatterns) {
const match = runtimeText.match(pattern);
if (match) {
totalMinutes += parseInt(match[1]) * 60;
break;
}
}
// Try each minute pattern until one matches
for (const pattern of langConfig.scraping.runtimeMinutePatterns) {
const match = runtimeText.match(pattern);
if (match) {
totalMinutes += parseInt(match[1]);
break;
}
if (minutesMatch) {
totalMinutes += parseInt(minutesMatch[1]);
}
return totalMinutes > 0 ? totalMinutes : undefined;
+12
View File
@@ -640,6 +640,18 @@ export class ProwlarrService {
// Singleton instance
let prowlarrService: ProwlarrService | null = null;
/**
* Invalidate the cached ProwlarrService singleton.
* Must be called after updating Prowlarr URL or API key so that
* background jobs (search, RSS monitor, etc.) pick up the new credentials.
*/
export function invalidateProwlarrService(): void {
if (prowlarrService) {
logger.info('Prowlarr service singleton invalidated — will reconnect with new credentials on next use');
}
prowlarrService = null;
}
export async function getProwlarrService(): Promise<ProwlarrService> {
if (!prowlarrService) {
// Get configuration from database
+16 -22
View File
@@ -27,12 +27,10 @@ const parseTorrent = (parseTorrentModule as any).default || parseTorrentModule;
const logger = RMABLogger.create('QBittorrent');
export interface AddTorrentOptions {
savePath?: string;
category?: string;
tags?: string[];
paused?: boolean;
skipChecking?: boolean;
sequentialDownload?: boolean;
}
export interface TorrentInfo {
@@ -276,7 +274,7 @@ export class QBittorrentService implements IDownloadClient {
/**
* Add magnet link - hash is extractable from URI (deterministic)
*/
private async addMagnetLink(
protected async addMagnetLink(
magnetUrl: string,
category: string,
options?: AddTorrentOptions
@@ -299,20 +297,18 @@ export class QBittorrentService implements IDownloadClient {
// Torrent doesn't exist, continue with adding
}
// Apply reverse path mapping (local → remote) to savepath
const localSavePath = options?.savePath || this.defaultSavePath;
const remoteSavePath = PathMapper.reverseTransform(localSavePath, this.pathMappingConfig);
// Upload via 'urls' parameter
// Set ratioLimit and seedingTimeLimit to -1 (unlimited) so qBittorrent's
// Note: savepath is intentionally omitted — the category (managed by ensureCategory)
// defines the save path. Omitting per-torrent savepath allows qBittorrent to use
// Automatic Torrent Management, respecting the user's "incomplete downloads" temp folder.
// sequentialDownload is also omitted — left to qBittorrent's own settings.
// ratioLimit and seedingTimeLimit are set to -1 (unlimited) so qBittorrent's
// global seeding rules don't remove the torrent prematurely.
// RMAB manages torrent lifecycle via the cleanup-seeded-torrents processor.
const form = new URLSearchParams({
urls: magnetUrl,
savepath: remoteSavePath,
category,
paused: options?.paused ? 'true' : 'false',
sequentialDownload: (options?.sequentialDownload !== false).toString(),
ratioLimit: '-1',
seedingTimeLimit: '-1',
});
@@ -341,7 +337,7 @@ export class QBittorrentService implements IDownloadClient {
/**
* Add .torrent file - download, parse, extract hash, upload content (deterministic)
*/
private async addTorrentFile(
protected async addTorrentFile(
torrentUrl: string,
category: string,
options?: AddTorrentOptions
@@ -446,11 +442,13 @@ export class QBittorrentService implements IDownloadClient {
// Torrent doesn't exist, continue with adding
}
// Apply reverse path mapping (local → remote) to savepath
const localSavePath = options?.savePath || this.defaultSavePath;
const remoteSavePath = PathMapper.reverseTransform(localSavePath, this.pathMappingConfig);
// Upload .torrent file content via multipart/form-data
// Note: savepath is intentionally omitted — the category (managed by ensureCategory)
// defines the save path. Omitting per-torrent savepath allows qBittorrent to use
// Automatic Torrent Management, respecting the user's "incomplete downloads" temp folder.
// sequentialDownload is also omitted — left to qBittorrent's own settings.
// ratioLimit and seedingTimeLimit override qBittorrent's global seeding rules —
// RMAB manages torrent lifecycle via the cleanup-seeded-torrents processor.
const formData = new FormData();
const filename = parsedTorrent.name ? `${parsedTorrent.name}.torrent` : 'torrent.torrent';
@@ -458,11 +456,8 @@ export class QBittorrentService implements IDownloadClient {
filename,
contentType: 'application/x-bittorrent',
});
formData.append('savepath', remoteSavePath);
formData.append('category', category);
formData.append('paused', options?.paused ? 'true' : 'false');
formData.append('sequentialDownload', (options?.sequentialDownload !== false).toString());
// Override qBittorrent's global seeding rules — RMAB manages torrent lifecycle
formData.append('ratioLimit', '-1');
formData.append('seedingTimeLimit', '-1');
@@ -494,7 +489,7 @@ export class QBittorrentService implements IDownloadClient {
* Checks existing categories first, then creates or updates as needed
* Applies reverse path mapping (local remote) for remote seedbox scenarios
*/
private async ensureCategory(category: string): Promise<void> {
protected async ensureCategory(category: string): Promise<void> {
if (!this.cookie) {
await this.login();
}
@@ -1013,7 +1008,6 @@ export class QBittorrentService implements IDownloadClient {
category: options?.category,
paused: options?.paused,
tags: ['audiobook'],
sequentialDownload: true,
});
}
@@ -1081,7 +1075,7 @@ export class QBittorrentService implements IDownloadClient {
/**
* Map a TorrentInfo object to the unified DownloadInfo format.
*/
private mapTorrentToDownloadInfo(torrent: TorrentInfo): DownloadInfo {
protected mapTorrentToDownloadInfo(torrent: TorrentInfo): DownloadInfo {
return {
id: torrent.hash,
name: torrent.name,
@@ -1194,7 +1188,7 @@ export class QBittorrentService implements IDownloadClient {
/**
* Extract info_hash from magnet link
*/
private extractHashFromMagnet(magnetUrl: string): string | null {
protected extractHashFromMagnet(magnetUrl: string): string | null {
// Extract hash from magnet:?xt=urn:btih:HASH
const match = magnetUrl.match(/xt=urn:btih:([a-fA-F0-9]{40}|[a-zA-Z0-9]{32})/i);
if (match) {
+105
View File
@@ -0,0 +1,105 @@
/**
* Component: RDT-Client Integration Service
* Documentation: documentation/phase3/download-clients.md
*
* RDT-Client is a Real-Debrid torrent proxy that emulates the qBittorrent API.
* Extends QBittorrentService and overrides behavioral differences:
* - Duplicate detection: deletes stale torrent before adding fresh (no false matches)
* - postProcess: removes torrent entry from client after files are organized
* - ensureCategory: no-op (RDT-Client doesn't support categories)
*/
import { RMABLogger } from '../utils/logger';
import { DownloadClientType } from '../interfaces/download-client.interface';
import { QBittorrentService, AddTorrentOptions } from './qbittorrent.service';
const logger = RMABLogger.create('RDTClient');
export class RDTClientService extends QBittorrentService {
override readonly clientType: DownloadClientType = 'rdtclient';
/**
* Override: Delete any existing torrent with the same hash before adding.
* RDT-Client can have stale entries from previous requests that cause
* false duplicate detection always start fresh.
*/
protected override async addMagnetLink(
magnetUrl: string,
category: string,
options?: AddTorrentOptions
): Promise<string> {
const infoHash = this.extractHashFromMagnet(magnetUrl);
if (infoHash) {
await this.deleteStaleIfExists(infoHash);
}
return super.addMagnetLink(magnetUrl, category, options);
}
/**
* Override: Delete any existing torrent with the same hash before adding.
* Same rationale as addMagnetLink prevent false duplicate short-circuits.
*/
protected override async addTorrentFile(
torrentUrl: string,
category: string,
options?: AddTorrentOptions
): Promise<string> {
// We can't pre-extract the hash from a .torrent URL without downloading it,
// so we let the parent handle the full flow. The parent's duplicate check
// calls getTorrent which will find any stale entry — but the parent
// short-circuits on duplicates. To handle this, we override addTorrentFile
// to intercept after the parent downloads and parses the torrent.
//
// The parent's addTorrentFile downloads the .torrent, parses it, checks for
// duplicates, then uploads. Since we can't hook into the middle of that flow
// without duplicating the download logic, we accept that .torrent file adds
// may encounter a stale duplicate. The primary use case (magnet links from
// indexers) is handled by the addMagnetLink override above.
//
// For .torrent files, the parent will return the existing hash if a duplicate
// is found. The postProcess cleanup after organize will still clean it up.
return super.addTorrentFile(torrentUrl, category, options);
}
/**
* Override: Remove torrent entry from RDT-Client after files are organized.
* Unlike qBittorrent (which seeds), RDT-Client torrents should be cleaned up
* immediately Real-Debrid handles seeding on their infrastructure.
*/
override async postProcess(id: string): Promise<void> {
try {
logger.info(`Removing torrent ${id} from RDT-Client (post-organize cleanup)`);
await this.deleteTorrent(id, false);
logger.info(`Successfully removed torrent ${id} from RDT-Client`);
} catch (error) {
// Non-fatal: torrent may already have been removed
logger.warn(
`Failed to remove torrent ${id} from RDT-Client: ${error instanceof Error ? error.message : String(error)}`
);
}
}
/**
* Override: No-op. RDT-Client doesn't support qBittorrent categories.
* Avoids 404 errors that appear in logs when the parent tries to create/update categories.
*/
protected override async ensureCategory(_category: string): Promise<void> {
// No-op: RDT-Client does not support categories
}
/**
* Delete a stale torrent if it exists, so a fresh add doesn't short-circuit.
*/
private async deleteStaleIfExists(hash: string): Promise<void> {
try {
await this.getTorrent(hash);
// If we get here, torrent exists — delete it
logger.info(`Deleting stale torrent ${hash} from RDT-Client before fresh add`);
await this.deleteTorrent(hash, false);
} catch {
// Torrent doesn't exist — nothing to clean up
}
}
}
@@ -11,7 +11,7 @@
// =========================================================================
/** Supported download client types — single source of truth */
export const SUPPORTED_CLIENT_TYPES = ['qbittorrent', 'sabnzbd', 'nzbget', 'transmission'] as const;
export const SUPPORTED_CLIENT_TYPES = ['qbittorrent', 'sabnzbd', 'nzbget', 'transmission', 'rdtclient'] as const;
/** Identifies the specific download client software */
export type DownloadClientType = (typeof SUPPORTED_CLIENT_TYPES)[number];
@@ -22,6 +22,7 @@ export const CLIENT_DISPLAY_NAMES: Record<DownloadClientType, string> = {
sabnzbd: 'SABnzbd',
nzbget: 'NZBGet',
transmission: 'Transmission',
rdtclient: 'RDT-Client',
};
/** Get display name for a client type, falling back to the raw type */
@@ -38,6 +39,7 @@ export const CLIENT_PROTOCOL_MAP: Record<DownloadClientType, ProtocolType> = {
sabnzbd: 'usenet',
nzbget: 'usenet',
transmission: 'torrent',
rdtclient: 'torrent',
};
/** Unified download status across all clients */
@@ -316,6 +316,7 @@ async function downloadFileWithProgress(
let bytesDownloaded = 0;
let lastLogTime = Date.now();
let lastDbUpdateTime = Date.now();
let dbUpdatePending = false; // Guard against stacking unresolved DB updates
response.data.on('data', (chunk: Buffer) => {
bytesDownloaded += chunk.length;
@@ -332,18 +333,18 @@ async function downloadFileWithProgress(
logger.info(`Download progress: ${percent}% (${(bytesDownloaded / (1024 * 1024)).toFixed(1)} MB, ${speedMBps.toFixed(2)} MB/s)`);
lastLogTime = now;
// Update database with progress (non-blocking)
if (now - lastDbUpdateTime >= PROGRESS_UPDATE_INTERVAL_MS) {
// Update database with progress (non-blocking, at most 1 in-flight at a time)
if (now - lastDbUpdateTime >= PROGRESS_UPDATE_INTERVAL_MS && !dbUpdatePending) {
lastDbUpdateTime = now;
dbUpdatePending = true;
// Non-blocking update - fire and forget
prisma.request.update({
where: { id: tracking.requestId },
data: {
progress: Math.min(percent, 99), // Cap at 99% until fully complete
updatedAt: new Date(),
},
}).catch(() => {}); // Ignore errors during progress update
}).catch(() => {}).finally(() => { dbUpdatePending = false; });
}
}
});
@@ -16,8 +16,23 @@ import { CLIENT_PROTOCOL_MAP, DownloadClientType } from '../interfaces/download-
* Checks download progress from download client and updates request status
* Re-schedules itself if download is still in progress
*/
/** Base polling interval in seconds */
const BASE_POLL_INTERVAL = 10;
/** Maximum polling interval in seconds (5 minutes) */
const MAX_POLL_INTERVAL = 300;
/**
* Compute next poll delay with exponential backoff for stalled downloads.
* Active downloads poll every 10s; stalled downloads back off up to 5 min.
*/
function getBackoffDelay(stallCount: number): number {
if (stallCount <= 0) return BASE_POLL_INTERVAL;
return Math.min(BASE_POLL_INTERVAL * Math.pow(2, stallCount), MAX_POLL_INTERVAL);
}
export async function processMonitorDownload(payload: MonitorDownloadPayload): Promise<any> {
const { requestId, downloadHistoryId, downloadClientId, downloadClient, jobId } = payload;
const { requestId, downloadHistoryId, downloadClientId, downloadClient, jobId,
lastProgress: prevProgress, stallCount: prevStallCount } = payload;
const logger = RMABLogger.forJob(jobId, 'MonitorDownload');
@@ -199,22 +214,35 @@ export async function processMonitorDownload(payload: MonitorDownloadPayload): P
progress: progressPercent,
};
} else {
// Still downloading - schedule another check in 10 seconds
// Still downloading — compute adaptive poll interval
const isStalled = info.downloadSpeed === 0
|| progressPercent === (prevProgress ?? -1)
|| progressState === 'paused'
|| progressState === 'queued'
|| progressState === 'checking';
const stallCount = isStalled ? (prevStallCount ?? 0) + 1 : 0;
const delay = getBackoffDelay(stallCount);
const jobQueue = getJobQueueService();
await jobQueue.addMonitorJob(
requestId,
downloadHistoryId,
downloadClientId,
downloadClient,
10 // Delay 10 seconds between checks
delay,
progressPercent,
stallCount
);
// Only log every 5% progress to reduce log spam
const shouldLog = progressPercent % 5 === 0 || progressPercent < 5;
// Only log every 5% progress to reduce log spam, but always log stall transitions
const shouldLog = progressPercent % 5 === 0 || progressPercent < 5
|| (stallCount === 1) || (stallCount > 0 && stallCount % 10 === 0);
if (shouldLog) {
logger.info(`Request ${requestId}: ${progressPercent}% complete (${progressState})`, {
speed: info.downloadSpeed,
eta: info.eta,
...(stallCount > 0 && { stallCount, nextPollSec: delay }),
});
}
@@ -227,6 +255,8 @@ export async function processMonitorDownload(payload: MonitorDownloadPayload): P
speed: info.downloadSpeed,
eta: info.eta,
state: progressState,
stallCount,
nextPollSec: delay,
};
}
} catch (error) {
@@ -124,6 +124,9 @@ export async function processMonitorRssFeeds(payload: MonitorRssFeedsPayload): P
break;
}
}
// Spread DB operations over time to avoid connection pool exhaustion
await new Promise(resolve => setTimeout(resolve, 100));
}
logger.info(`RSS monitoring complete: ${matched} matches found and queued for processing`);
@@ -864,8 +864,13 @@ async function cleanupDownloadAfterOrganize(
removeAfterProcessing: indexer?.removeAfterProcessing ?? 'undefined',
});
// Check if this is a non-torrent indexer with cleanup enabled
if (!indexer || indexer.protocol?.toLowerCase() === 'torrent' || !indexer.removeAfterProcessing) {
// Check if this is a non-torrent indexer with cleanup enabled.
// RDT-Client is an exception: even though it's torrent protocol, it needs cleanup
// because Real-Debrid handles seeding — local torrent entries should be removed.
const isRDTClient = downloadHistory.downloadClient === 'rdtclient';
const isTorrentProtocol = indexer?.protocol?.toLowerCase() === 'torrent';
if (!indexer || (!isRDTClient && isTorrentProtocol) || !indexer.removeAfterProcessing) {
return;
}
@@ -157,6 +157,9 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
);
triggered++;
logger.info(`Triggered organize job for ${request.type || 'audiobook'} request ${request.id}: ${request.audiobook.title}`);
// Spread DB operations over time to avoid connection pool exhaustion
await new Promise(resolve => setTimeout(resolve, 100));
} catch (error) {
logger.error(`Failed to trigger organize for request ${request.id}: ${error instanceof Error ? error.message : 'Unknown error'}`);
skipped++;
@@ -44,6 +44,7 @@ export async function processRetryMissingTorrents(payload: RetryMissingTorrentsP
}
// Trigger appropriate search job for each request based on type
// Throttle: 100ms delay between jobs to avoid connection pool burst
const jobQueue = getJobQueueService();
let triggered = 0;
@@ -73,6 +74,9 @@ export async function processRetryMissingTorrents(payload: RetryMissingTorrentsP
} catch (error) {
logger.error(`Failed to trigger search for request ${request.id}: ${error instanceof Error ? error.message : 'Unknown error'}`);
}
// Spread DB operations over time to avoid connection pool exhaustion
await new Promise(resolve => setTimeout(resolve, 100));
}
logger.info(`Triggered ${triggered}/${requests.length} search jobs`);
+15 -2
View File
@@ -14,6 +14,8 @@ import { RMABLogger } from '../utils/logger';
import { getProwlarrService } from '../integrations/prowlarr.service';
import { rankEbookTorrents, RankedEbookTorrent } from '../utils/ranking-algorithm';
import { groupIndexersByCategories, getGroupDescription } from '../utils/indexer-grouping';
import { getLanguageForRegion } from '../constants/language-config';
import type { AudibleRegion } from '../types/audible';
// Import ebook scraper functions for Anna's Archive
import {
@@ -151,6 +153,11 @@ async function searchAnnasArchive(
const baseUrl = await configService.get('ebook_sidecar_base_url') || 'https://annas-archive.li';
const flaresolverrUrl = await configService.get('ebook_sidecar_flaresolverr_url') || undefined;
// Get language code from Audible region config
const region = await configService.getAudibleRegion() as AudibleRegion;
const langConfig = getLanguageForRegion(region);
const languageCode = langConfig.annasArchiveLang;
if (flaresolverrUrl) {
logger.info(`Using FlareSolverr at ${flaresolverrUrl}`);
}
@@ -161,7 +168,7 @@ async function searchAnnasArchive(
// Try ASIN search first (exact match - best)
if (audiobook.asin) {
logger.info(`Searching Anna's Archive by ASIN: ${audiobook.asin} (format: ${preferredFormat})...`);
md5 = await searchByAsin(audiobook.asin, preferredFormat, baseUrl, logger, flaresolverrUrl);
md5 = await searchByAsin(audiobook.asin, preferredFormat, baseUrl, logger, flaresolverrUrl, languageCode);
if (md5) {
logger.info(`Found via ASIN: ${md5}`);
@@ -174,7 +181,7 @@ async function searchAnnasArchive(
// Fallback to title + author search
if (!md5) {
logger.info(`Searching Anna's Archive by title + author: "${audiobook.title}" by ${audiobook.author}...`);
md5 = await searchByTitle(audiobook.title, audiobook.author, preferredFormat, baseUrl, logger, flaresolverrUrl);
md5 = await searchByTitle(audiobook.title, audiobook.author, preferredFormat, baseUrl, logger, flaresolverrUrl, languageCode);
if (md5) {
logger.info(`Found via title search: ${md5}`);
@@ -301,6 +308,10 @@ async function searchIndexers(
logger.info(`Will filter ${aboveThreshold.length} results > 20 MB (too large for ebooks)`);
}
// Get language-specific stop words for ranking
const ebookRegion = await configService.getAudibleRegion() as AudibleRegion;
const ebookLangConfig = getLanguageForRegion(ebookRegion);
// Rank results with ebook-specific scoring
// This filters out > 20MB and uses inverted size scoring
const rankedResults = rankEbookTorrents(allResults, {
@@ -311,6 +322,8 @@ async function searchIndexers(
indexerPriorities,
flagConfigs,
requireAuthor: true, // Automatic mode - prevent wrong authors
stopWords: ebookLangConfig.stopWords,
characterReplacements: ebookLangConfig.characterReplacements,
});
// Log filter results
@@ -9,6 +9,8 @@ import { getProwlarrService } from '../integrations/prowlarr.service';
import { getRankingAlgorithm } from '../utils/ranking-algorithm';
import { groupIndexersByCategories, getGroupDescription } from '../utils/indexer-grouping';
import { RMABLogger } from '../utils/logger';
import { getLanguageForRegion } from '../constants/language-config';
import type { AudibleRegion } from '../types/audible';
/**
* Process search indexers job
@@ -146,8 +148,10 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
logger.info(`Will filter ${belowThreshold.length} results < ${sizeMBThreshold} MB (likely ebooks)`);
}
// Get ranking algorithm
// Get ranking algorithm and language-specific stop words
const ranker = getRankingAlgorithm();
const region = await configService.getAudibleRegion() as AudibleRegion;
const langConfig = getLanguageForRegion(region);
// Rank results with indexer priorities and flag configs
// Note: rankTorrents now filters out results < 20 MB internally
@@ -159,7 +163,9 @@ export async function processSearchIndexers(payload: SearchIndexersPayload): Pro
}, {
indexerPriorities,
flagConfigs,
requireAuthor: true // Automatic mode - prevent wrong authors
requireAuthor: true, // Automatic mode - prevent wrong authors
stopWords: langConfig.stopWords,
characterReplacements: langConfig.characterReplacements,
});
// Log filter results
@@ -16,6 +16,7 @@ import { QBittorrentService } from '@/lib/integrations/qbittorrent.service';
import { SABnzbdService } from '@/lib/integrations/sabnzbd.service';
import { NZBGetService } from '@/lib/integrations/nzbget.service';
import { TransmissionService } from '@/lib/integrations/transmission.service';
import { RDTClientService } from '@/lib/integrations/rdtclient.service';
import { PathMappingConfig } from '@/lib/utils/path-mapper';
import { IDownloadClient, DownloadClientType, ProtocolType, CLIENT_PROTOCOL_MAP, getClientDisplayName } from '@/lib/interfaces/download-client.interface';
@@ -193,6 +194,8 @@ export class DownloadClientManager {
return this.createNZBGetService(config, downloadDir);
case 'transmission':
return this.createTransmissionService(config, downloadDir);
case 'rdtclient':
return this.createRDTClientService(config, downloadDir);
default:
throw new Error(`Unsupported download client type: ${config.type}`);
}
@@ -335,6 +338,29 @@ export class DownloadClientManager {
);
}
/**
* Create RDT-Client service instance (same constructor as qBittorrent identical API)
*/
private createRDTClientService(config: DownloadClientConfig, downloadDir: string): RDTClientService {
const pathMapping: PathMappingConfig | undefined = config.remotePathMappingEnabled && config.remotePath && config.localPath
? {
enabled: true,
remotePath: config.remotePath,
localPath: config.localPath,
}
: undefined;
return new RDTClientService(
config.url,
config.username || '',
config.password || '',
downloadDir,
config.category || 'readmeabook',
config.disableSSLVerify,
pathMapping
);
}
/**
* Migrate legacy single-client config to new multi-client format
*/
+13 -10
View File
@@ -170,7 +170,8 @@ export async function downloadEbook(
preferredFormat: string = 'epub',
baseUrl: string = 'https://annas-archive.li',
logger?: RMABLogger,
flaresolverrUrl?: string
flaresolverrUrl?: string,
languageCode: string = 'en'
): Promise<EbookDownloadResult> {
try {
let md5: string | null = null;
@@ -183,7 +184,7 @@ export async function downloadEbook(
// Step 1: Try ASIN search (exact match - best)
if (asin) {
await logger?.info(`Searching by ASIN: ${asin} (format: ${preferredFormat})...`);
md5 = await searchByAsin(asin, preferredFormat, baseUrl, logger, flaresolverrUrl);
md5 = await searchByAsin(asin, preferredFormat, baseUrl, logger, flaresolverrUrl, languageCode);
if (md5) {
await logger?.info(`Found via ASIN: ${md5}`);
@@ -195,7 +196,7 @@ export async function downloadEbook(
// Step 2: Fallback to title + author search
if (!md5) {
await logger?.info(`Searching by title + author: "${title}" by ${author}...`);
md5 = await searchByTitle(title, author, preferredFormat, baseUrl, logger, flaresolverrUrl);
md5 = await searchByTitle(title, author, preferredFormat, baseUrl, logger, flaresolverrUrl, languageCode);
if (md5) {
await logger?.info(`Found via title search: ${md5}`);
@@ -312,10 +313,11 @@ export async function searchByAsin(
format: string,
baseUrl: string,
logger?: RMABLogger,
flaresolverrUrl?: string
flaresolverrUrl?: string,
languageCode: string = 'en'
): Promise<string | null> {
// Check cache first
const cacheKey = `${asin}-${format}`;
const cacheKey = `${asin}-${format}-${languageCode}`;
if (md5Cache.has(cacheKey)) {
const cached = md5Cache.get(cacheKey);
if (cached) {
@@ -327,7 +329,7 @@ export async function searchByAsin(
try {
// Build search URL with ASIN and optional format filter
const formatParam = format && format !== 'any' ? `ext=${format}&` : '';
const searchUrl = `${baseUrl}/search?${formatParam}lang=en&q=%22asin:${asin}%22`;
const searchUrl = `${baseUrl}/search?${formatParam}lang=${languageCode}&q=%22asin:${asin}%22`;
moduleLogger.debug(`ASIN search URL: ${searchUrl}`);
@@ -404,10 +406,11 @@ export async function searchByTitle(
format: string,
baseUrl: string,
logger?: RMABLogger,
flaresolverrUrl?: string
flaresolverrUrl?: string,
languageCode: string = 'en'
): Promise<string | null> {
// Check cache first
const cacheKey = `title-${title}-${author}-${format}`.toLowerCase();
const cacheKey = `title-${title}-${author}-${format}-${languageCode}`.toLowerCase();
if (md5Cache.has(cacheKey)) {
const cached = md5Cache.get(cacheKey);
if (cached) {
@@ -432,8 +435,8 @@ export async function searchByTitle(
// Add content type filters (books only, all fiction/nonfiction/unknown)
searchUrl += '&content=book_nonfiction&content=book_fiction&content=book_unknown';
// Add language filter (English)
searchUrl += '&lang=en';
// Add language filter
searchUrl += `&lang=${languageCode}`;
// Empty raw query (we're using specific terms instead)
searchUrl += '&q=';
+14 -8
View File
@@ -63,6 +63,8 @@ export interface MonitorDownloadPayload extends JobPayload {
downloadHistoryId: string;
downloadClientId: string;
downloadClient: DownloadClientType;
lastProgress?: number; // Previous poll's progress (0-100) for stall detection
stallCount?: number; // Consecutive polls with no progress change (drives backoff)
}
export interface OrganizeFilesPayload extends JobPayload {
@@ -277,19 +279,19 @@ export class JobQueueService {
*/
private startProcessors(): void {
// Search indexers processor
this.queue.process('search_indexers', 3, async (job: BullJob<SearchIndexersPayload>) => {
this.queue.process('search_indexers', 2, async (job: BullJob<SearchIndexersPayload>) => {
const { processSearchIndexers } = await import('../processors/search-indexers.processor');
return await processSearchIndexers(job.data);
});
// Download torrent processor
this.queue.process('download_torrent', 3, async (job: BullJob<DownloadTorrentPayload>) => {
this.queue.process('download_torrent', 2, async (job: BullJob<DownloadTorrentPayload>) => {
const { processDownloadTorrent } = await import('../processors/download-torrent.processor');
return await processDownloadTorrent(job.data);
});
// Monitor download processor
this.queue.process('monitor_download', 5, async (job: BullJob<MonitorDownloadPayload>) => {
this.queue.process('monitor_download', 2, async (job: BullJob<MonitorDownloadPayload>) => {
const { processMonitorDownload } = await import('../processors/monitor-download.processor');
return await processMonitorDownload(job.data);
});
@@ -357,23 +359,23 @@ export class JobQueueService {
});
// Send notification processor
this.queue.process('send_notification', 5, async (job: BullJob<SendNotificationPayload>) => {
this.queue.process('send_notification', 2, async (job: BullJob<SendNotificationPayload>) => {
const { processSendNotification } = await import('../processors/send-notification.processor');
return await processSendNotification(job.data);
});
// Ebook-specific processors
this.queue.process('search_ebook', 3, async (job: BullJob<SearchEbookPayload>) => {
this.queue.process('search_ebook', 2, async (job: BullJob<SearchEbookPayload>) => {
const { processSearchEbook } = await import('../processors/search-ebook.processor');
return await processSearchEbook(job.data);
});
this.queue.process('start_direct_download', 3, async (job: BullJob<StartDirectDownloadPayload>) => {
this.queue.process('start_direct_download', 2, async (job: BullJob<StartDirectDownloadPayload>) => {
const { processStartDirectDownload } = await import('../processors/direct-download.processor');
return await processStartDirectDownload(job.data);
});
this.queue.process('monitor_direct_download', 5, async (job: BullJob<MonitorDirectDownloadPayload>) => {
this.queue.process('monitor_direct_download', 2, async (job: BullJob<MonitorDirectDownloadPayload>) => {
const { processMonitorDirectDownload } = await import('../processors/direct-download.processor');
return await processMonitorDirectDownload(job.data);
});
@@ -563,7 +565,9 @@ export class JobQueueService {
downloadHistoryId: string,
downloadClientId: string,
downloadClient: DownloadClientType,
delaySeconds: number = 0
delaySeconds: number = 0,
lastProgress?: number,
stallCount?: number
): Promise<string> {
return await this.addJob(
'monitor_download',
@@ -572,6 +576,8 @@ export class JobQueueService {
downloadHistoryId,
downloadClientId,
downloadClient,
lastProgress,
stallCount,
} as MonitorDownloadPayload,
{
priority: 5, // Medium priority
@@ -84,6 +84,7 @@ export async function createRequestForUser(
let year: number | undefined;
let series: string | undefined;
let seriesPart: string | undefined;
let seriesAsin: string | undefined;
try {
const audibleService = getAudibleService();
const audnexusData = await audibleService.getAudiobookDetails(audiobook.asin);
@@ -100,6 +101,7 @@ export async function createRequestForUser(
}
if (audnexusData?.series) series = audnexusData.series;
if (audnexusData?.seriesPart) seriesPart = audnexusData.seriesPart;
if (audnexusData?.seriesAsin) seriesAsin = audnexusData.seriesAsin;
} catch (error) {
logger.warn(`Failed to fetch Audnexus data for ASIN ${audiobook.asin}: ${error instanceof Error ? error.message : 'Unknown error'}`);
}
@@ -121,6 +123,7 @@ export async function createRequestForUser(
year,
series,
seriesPart,
seriesAsin,
status: 'requested',
},
});
@@ -134,6 +137,7 @@ export async function createRequestForUser(
if (year) updates.year = year;
if (series) updates.series = series;
if (seriesPart) updates.seriesPart = seriesPart;
if (seriesAsin) updates.seriesAsin = seriesAsin;
if (Object.keys(updates).length > 0) {
audiobookRecord = await prisma.audiobook.update({
+31 -3
View File
@@ -51,12 +51,18 @@ export class SchedulerService {
logger.info('Initializing scheduler service...');
// Re-encrypt any notification backends with plaintext sensitive fields
try {
await getNotificationService().reEncryptUnprotectedBackends();
} catch (error) {
logger.error('Failed to re-encrypt notification backends (non-fatal)', {
error: error instanceof Error ? error.message : String(error),
});
}
// Create default jobs if they don't exist
await this.ensureDefaultJobs();
// Load and schedule all enabled jobs
// Load and schedule all enabled jobs (works with whatever jobs exist in DB)
await this.scheduleAllJobs();
// Check and trigger overdue jobs
@@ -66,7 +72,8 @@ export class SchedulerService {
}
/**
* Ensure default jobs exist in database
* Ensure default jobs exist in database.
* Each job is created independently so a single failure doesn't block the rest.
*/
private async ensureDefaultJobs(): Promise<void> {
const defaults = [
@@ -128,7 +135,11 @@ export class SchedulerService {
},
];
let created = 0;
let failed = 0;
for (const defaultJob of defaults) {
try {
const existing = await prisma.scheduledJob.findFirst({
where: { type: defaultJob.type },
});
@@ -137,8 +148,22 @@ export class SchedulerService {
await prisma.scheduledJob.create({
data: defaultJob,
});
logger.info(`Created default job: ${defaultJob.name} (disabled by default)`);
created++;
logger.info(`Created default job: ${defaultJob.name} (enabled: ${defaultJob.enabled})`);
}
} catch (error) {
failed++;
logger.error(`Failed to create default job: ${defaultJob.name}`, {
type: defaultJob.type,
error: error instanceof Error ? error.message : String(error),
});
}
}
if (failed > 0) {
logger.warn(`Default jobs: ${created} created, ${failed} failed — failed jobs will be retried on next restart`);
} else if (created > 0) {
logger.info(`Default jobs: ${created} created`);
}
}
@@ -466,6 +491,9 @@ export class SchedulerService {
if (this.isJobOverdue(job)) {
logger.info(`Job "${job.name}" is overdue, triggering now...`);
await this.triggerJobNow(job.id);
// Stagger triggers to avoid connection pool burst on startup
await new Promise(resolve => setTimeout(resolve, 500));
}
} catch (error) {
logger.error(`Failed to trigger overdue job "${job.name}"`, { error: error instanceof Error ? error.message : String(error) });
+10 -8
View File
@@ -3,6 +3,8 @@
* Documentation: documentation/integrations/audible.md
*/
import type { SupportedLanguage } from '../constants/language-config';
export type AudibleRegion = 'us' | 'ca' | 'uk' | 'au' | 'in' | 'de' | 'es';
export interface AudibleRegionConfig {
@@ -10,7 +12,7 @@ export interface AudibleRegionConfig {
name: string;
baseUrl: string;
audnexusParam: string;
isEnglish: boolean;
language: SupportedLanguage;
}
export const AUDIBLE_REGIONS: Record<AudibleRegion, AudibleRegionConfig> = {
@@ -19,49 +21,49 @@ export const AUDIBLE_REGIONS: Record<AudibleRegion, AudibleRegionConfig> = {
name: 'United States',
baseUrl: 'https://www.audible.com',
audnexusParam: 'us',
isEnglish: true,
language: 'en',
},
ca: {
code: 'ca',
name: 'Canada',
baseUrl: 'https://www.audible.ca',
audnexusParam: 'ca',
isEnglish: true,
language: 'en',
},
uk: {
code: 'uk',
name: 'United Kingdom',
baseUrl: 'https://www.audible.co.uk',
audnexusParam: 'uk',
isEnglish: true,
language: 'en',
},
au: {
code: 'au',
name: 'Australia',
baseUrl: 'https://www.audible.com.au',
audnexusParam: 'au',
isEnglish: true,
language: 'en',
},
in: {
code: 'in',
name: 'India',
baseUrl: 'https://www.audible.in',
audnexusParam: 'in',
isEnglish: true,
language: 'en',
},
de: {
code: 'de',
name: 'Germany',
baseUrl: 'https://www.audible.de',
audnexusParam: 'de',
isEnglish: false,
language: 'de',
},
es: {
code: 'es',
name: 'Spain',
baseUrl: 'https://www.audible.es',
audnexusParam: 'es',
isEnglish: false,
language: 'es',
}
};
+14 -1
View File
@@ -163,7 +163,20 @@ export async function enrichAudiobooksWithMatches(
audiobooks: Array<AudiobookMatchInput & Record<string, any>>,
userId?: string
) {
const results = await Promise.all(audiobooks.map((book) => enrichAudiobookWithMatch(book)));
// Batch parallel DB queries to avoid connection pool exhaustion
const BATCH_SIZE = 5;
const results: Awaited<ReturnType<typeof enrichAudiobookWithMatch>>[] = [];
for (let i = 0; i < audiobooks.length; i += BATCH_SIZE) {
const batch = audiobooks.slice(i, i + BATCH_SIZE);
const batchResults = await Promise.allSettled(batch.map((book) => enrichAudiobookWithMatch(book)));
for (const result of batchResults) {
if (result.status === 'fulfilled') {
results.push(result.value);
} else {
logger.error('Failed to enrich audiobook', { error: result.reason instanceof Error ? result.reason.message : String(result.reason) });
}
}
}
// Always enrich with request status (check ANY user's requests)
const asins = audiobooks.map(book => book.asin);
+50 -18
View File
@@ -40,6 +40,8 @@ export interface RankTorrentsOptions {
indexerPriorities?: Map<number, number>; // indexerId -> priority (1-25)
flagConfigs?: IndexerFlagConfig[]; // Flag bonus configurations
requireAuthor?: boolean; // Enforce author presence check (default: true)
stopWords?: string[]; // Language-specific stop words for matching
characterReplacements?: Record<string, string>; // Language-specific char replacements (e.g. ß→ss)
}
export interface EbookTorrentRequest {
@@ -52,6 +54,8 @@ export interface RankEbookTorrentsOptions {
indexerPriorities?: Map<number, number>; // indexerId -> priority (1-25)
flagConfigs?: IndexerFlagConfig[]; // Flag bonus configurations
requireAuthor?: boolean; // Enforce author presence check (default: true)
stopWords?: string[]; // Language-specific stop words for matching
characterReplacements?: Record<string, string>; // Language-specific char replacements (e.g. ß→ss)
}
export interface BonusModifier {
@@ -113,7 +117,9 @@ export class RankingAlgorithm {
const {
indexerPriorities,
flagConfigs,
requireAuthor = true // Safe default: require author in automatic mode
requireAuthor = true, // Safe default: require author in automatic mode
stopWords,
characterReplacements,
} = options;
// Filter out files < 20 MB (likely ebooks/samples)
const filteredTorrents = torrents.filter((torrent) => {
@@ -126,7 +132,7 @@ export class RankingAlgorithm {
const formatScore = this.scoreFormat(torrent);
const sizeScore = this.scoreSize(torrent, audiobook.durationMinutes);
const seederScore = this.scoreSeeders(torrent.seeders);
const matchScore = this.scoreMatch(torrent, audiobook, requireAuthor);
const matchScore = this.scoreMatch(torrent, audiobook, requireAuthor, stopWords, characterReplacements);
const baseScore = formatScore + sizeScore + seederScore + matchScore;
@@ -340,11 +346,22 @@ export class RankingAlgorithm {
* "Twelve.Months-Jim.Butcher" "twelve months jim butcher"
* "Author_Name_Book" "author name book"
*/
private normalizeForMatching(text: string): string {
return text
private normalizeForMatching(text: string, characterReplacements?: Record<string, string>): string {
let result = text
// Split CamelCase FIRST (before lowercasing): "TheCorrespondent" → "The Correspondent"
.replace(/([a-z])([A-Z])/g, '$1 $2')
.toLowerCase()
.toLowerCase();
// Apply language-specific character replacements before NFD (e.g. ß→ss)
if (characterReplacements) {
for (const [from, to] of Object.entries(characterReplacements)) {
result = result.replace(new RegExp(from, 'g'), to);
}
}
return result
// NFD normalization: convert accented chars to ASCII base forms
// e.g. "uber" from "uber", "senor" from "senor", "cafe" from "cafe"
.normalize('NFD')
.replace(/[\u0300-\u036f]/g, '')
// Replace underscores with spaces (must be explicit since \w includes _)
.replace(/_/g, ' ')
// Replace other punctuation/separators with spaces (preserves apostrophes in contractions)
@@ -362,11 +379,13 @@ export class RankingAlgorithm {
private scoreMatch(
torrent: TorrentResult,
audiobook: AudiobookRequest,
requireAuthor: boolean = true
requireAuthor: boolean = true,
customStopWords?: string[],
characterReplacements?: Record<string, string>
): number {
// Normalize for matching (handles CamelCase, punctuation separators)
const torrentTitle = this.normalizeForMatching(torrent.title);
const requestTitle = this.normalizeForMatching(audiobook.title);
// Normalize for matching (handles CamelCase, punctuation separators, diacritics)
const torrentTitle = this.normalizeForMatching(torrent.title, characterReplacements);
const requestTitle = this.normalizeForMatching(audiobook.title, characterReplacements);
// Parse authors from RAW string first (preserving commas for splitting)
// Then normalize individual authors for matching
@@ -377,19 +396,30 @@ export class RankingAlgorithm {
.filter(a => a.length > 2 && !['translator', 'narrator'].includes(a));
// Normalize parsed authors for matching (handles CamelCase in author names)
const normalizedAuthors = parsedAuthors.map(a => this.normalizeForMatching(a));
const normalizedAuthors = parsedAuthors.map(a => this.normalizeForMatching(a, characterReplacements));
// Combined normalized author string for fuzzy matching
const requestAuthorNormalized = normalizedAuthors.join(' ');
// ========== STAGE 1: WORD COVERAGE FILTER (MANDATORY) ==========
// Extract significant words (filter out common stop words)
const stopWords = ['the', 'a', 'an', 'of', 'on', 'in', 'at', 'by', 'for'];
// Use provided language-specific stop words, or fall back to English defaults
const stopWords = customStopWords || ['the', 'a', 'an', 'of', 'on', 'in', 'at', 'by', 'for'];
const extractWords = (text: string, stopList: string[]): string[] => {
return text
let processed = text
// Split CamelCase FIRST: "TheCorrespondent" → "The Correspondent"
.replace(/([a-z])([A-Z])/g, '$1 $2')
.toLowerCase()
.toLowerCase();
// Apply language-specific character replacements before NFD
if (characterReplacements) {
for (const [from, to] of Object.entries(characterReplacements)) {
processed = processed.replace(new RegExp(from, 'g'), to);
}
}
return processed
// NFD normalization for accented characters
.normalize('NFD')
.replace(/[\u0300-\u036f]/g, '')
// Replace underscores with spaces (must be explicit since \w includes _)
.replace(/_/g, ' ')
// Remove other punctuation (but keep apostrophes for contractions)
@@ -431,7 +461,7 @@ export class RankingAlgorithm {
}
// Normalize the required portion (handles CamelCase, punctuation)
const required = this.normalizeForMatching(requiredRaw);
const required = this.normalizeForMatching(requiredRaw, characterReplacements);
const optional = optionalMatches.join(' ');
return { required, optional };
@@ -653,7 +683,7 @@ export class RankingAlgorithm {
* @param requestAuthor - Raw author string (will be parsed and normalized internally)
* @returns true if at least ONE author is present with high confidence
*/
private checkAuthorPresence(torrentTitle: string, requestAuthor: string): boolean {
private checkAuthorPresence(torrentTitle: string, requestAuthor: string, characterReplacements?: Record<string, string>): boolean {
// Parse multiple authors (same logic as Stage 3 author matching)
const authors = requestAuthor
.split(/,|&| and | - /)
@@ -661,7 +691,7 @@ export class RankingAlgorithm {
.filter(a => a.length > 2 && !['translator', 'narrator'].includes(a));
// Normalize each author for matching
const normalizedAuthors = authors.map(a => this.normalizeForMatching(a));
const normalizedAuthors = authors.map(a => this.normalizeForMatching(a, characterReplacements));
return this.checkAuthorPresenceWithParsed(torrentTitle, normalizedAuthors);
}
@@ -788,7 +818,9 @@ export class RankingAlgorithm {
const {
indexerPriorities,
flagConfigs,
requireAuthor = true // Safe default: require author in automatic mode
requireAuthor = true, // Safe default: require author in automatic mode
stopWords,
characterReplacements,
} = options;
// Filter out files > 20 MB (too large for ebooks)
@@ -809,7 +841,7 @@ export class RankingAlgorithm {
const matchScore = this.scoreMatch(torrent, {
title: ebook.title,
author: ebook.author,
}, requireAuthor);
}, requireAuthor, stopWords, characterReplacements);
const baseScore = formatScore + sizeScore + seederScore + matchScore;
@@ -10,6 +10,7 @@ const prismaMock = createPrismaMock();
const audibleServiceMock = vi.hoisted(() => ({
search: vi.fn(),
getAudiobookDetails: vi.fn(),
getBaseUrl: vi.fn().mockReturnValue('https://www.audible.com'),
}));
const enrichMock = vi.hoisted(() => vi.fn());
const currentUserMock = vi.hoisted(() => vi.fn());
@@ -10,6 +10,7 @@ let authRequest: any;
const requireAuthMock = vi.hoisted(() => vi.fn());
const configServiceMock = vi.hoisted(() => ({
get: vi.fn(),
getAudibleRegion: vi.fn().mockResolvedValue('us'),
}));
const prowlarrMock = vi.hoisted(() => ({
search: vi.fn(),
@@ -43,6 +44,7 @@ vi.mock('@/lib/utils/indexer-grouping', () => ({
describe('Audiobooks search torrents route', () => {
beforeEach(() => {
vi.clearAllMocks();
configServiceMock.getAudibleRegion.mockResolvedValue('us');
authRequest = {
user: { id: 'user-1', role: 'user' },
json: vi.fn(),
+2 -1
View File
@@ -12,7 +12,7 @@ const prismaMock = createPrismaMock();
const requireAuthMock = vi.hoisted(() => vi.fn());
const prowlarrMock = vi.hoisted(() => ({ search: vi.fn(), searchWithVariations: vi.fn() }));
const rankTorrentsMock = vi.hoisted(() => vi.fn());
const configServiceMock = vi.hoisted(() => ({ get: vi.fn() }));
const configServiceMock = vi.hoisted(() => ({ get: vi.fn(), getAudibleRegion: vi.fn().mockResolvedValue('us') }));
const groupIndexersMock = vi.hoisted(() => vi.fn());
const groupDescriptionMock = vi.hoisted(() => vi.fn(() => 'Group'));
const configState = vi.hoisted(() => ({
@@ -75,6 +75,7 @@ vi.mock('fs/promises', () => ({ default: fsMock, ...fsMock, constants: { R_OK: 4
describe('Request action routes', () => {
beforeEach(() => {
vi.clearAllMocks();
configServiceMock.getAudibleRegion.mockResolvedValue('us');
configState.values.clear();
authRequest = { user: { id: 'user-1', role: 'user' }, json: vi.fn() };
requireAuthMock.mockImplementation((_req: any, handler: any) => handler(authRequest));
@@ -150,7 +150,9 @@ describe('processMonitorDownload', () => {
'dh-2',
'hash-2',
'qbittorrent',
10
10,
45, // progressPercent passed as lastProgress
0, // stallCount reset (download is actively progressing)
);
});
@@ -10,6 +10,7 @@ const prismaMock = createPrismaMock();
const configServiceMock = vi.hoisted(() => ({
get: vi.fn(),
getAudibleRegion: vi.fn().mockResolvedValue('us'),
}));
const jobQueueMock = vi.hoisted(() => ({
@@ -39,6 +40,7 @@ vi.mock('@/lib/services/ebook-scraper', () => ebookScraperMock);
describe('processSearchEbook', () => {
beforeEach(() => {
vi.clearAllMocks();
configServiceMock.getAudibleRegion.mockResolvedValue('us');
configServiceMock.get.mockImplementation(async (key: string) => {
if (key === 'ebook_sidecar_preferred_format') return 'epub';
if (key === 'ebook_sidecar_base_url') return 'https://annas-archive.li';
@@ -79,7 +81,8 @@ describe('processSearchEbook', () => {
'epub',
'https://annas-archive.li',
expect.anything(),
undefined
undefined,
'en'
);
expect(jobQueueMock.addStartDirectDownloadJob).toHaveBeenCalledWith(
'req-1',
@@ -123,7 +126,8 @@ describe('processSearchEbook', () => {
'epub',
'https://annas-archive.li',
expect.anything(),
undefined
undefined,
'en'
);
});
@@ -253,7 +257,8 @@ describe('processSearchEbook', () => {
'epub',
'https://annas-archive.li',
expect.anything(),
'http://flaresolverr:8191'
'http://flaresolverr:8191',
'en'
);
});
@@ -8,7 +8,7 @@ import { createPrismaMock } from '../helpers/prisma';
import { createJobQueueMock } from '../helpers/job-queue';
const prismaMock = createPrismaMock();
const configMock = vi.hoisted(() => ({ get: vi.fn() }));
const configMock = vi.hoisted(() => ({ get: vi.fn(), getAudibleRegion: vi.fn().mockResolvedValue('us') }));
const jobQueueMock = createJobQueueMock();
const prowlarrMock = vi.hoisted(() => ({ search: vi.fn(), searchWithVariations: vi.fn() }));
@@ -35,6 +35,7 @@ vi.mock('@/lib/integrations/audible.service', () => ({
describe('processSearchIndexers', () => {
beforeEach(() => {
vi.clearAllMocks();
configMock.getAudibleRegion.mockResolvedValue('us');
});
it('marks request awaiting_search when no results found', async () => {
+2 -2
View File
@@ -31,10 +31,10 @@
</Screenshots>
<Network>bridge</Network>
<Config Name="Web UI Host Port" Target="3030" Default="3030" Mode="tcp" Description="Port for ReadMeABook's web interface." Type="Port" Display="always" Required="true" Mask="false">3030</Config>
<Config Name="Appdata" Target="/app/config" Default="/mnt/user/appdata/readmeabook" Mode="rw" Description="Persistent config files" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/readmeabook</Config>
<Config Name="Config Location" Target="/app/config" Default="/mnt/user/appdata/readmeabook/config" Mode="rw" Description="Persistent config files" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/readmeabook/config</Config>
<Config Name="Download Location" Target="/downloads" Default="/mnt/user/data/downloads" Mode="rw" Description="Both your download client and RMAB must see files at the SAME path. See: https://github.com/kikootwo/ReadMeABook/blob/main/documentation/deployment/volume-mapping.md" Type="Path" Display="always" Required="true" Mask="false"/>
<Config Name="Media Library" Target="/media" Default="/mnt/user/data/media/audiobooks" Mode="rw" Description="Your audiobook/ebook library" Type="Path" Display="always" Required="true" Mask="false"/>
<Config Name="Postgres Storage Location" Target="/var/lib/postgresql/data" Default="readmeabook_pgdata" Mode="rw" Description="" Type="Path" Display="always" Required="true" Mask="false">readmeabook_pgdata</Config>
<Config Name="Postgres Storage Location" Target="/var/lib/postgresql/data" Default="/mnt/user/appdata/readmeabook/pgdata" Mode="rw" Description="" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/readmeabook/pgdata</Config>
<Config Name="Redis Storage Location" Target="/var/lib/redis" Default="/mnt/user/appdata/readmeabook/redis" Mode="rw" Description="" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/readmeabook/redis</Config>
<Config Name="App Cache" Target="/app/cache" Default="/mnt/user/appdata/readmeabook/cache" Mode="rw" Description="" Type="Path" Display="advanced" Required="true" Mask="false">/mnt/user/appdata/readmeabook/cache</Config>
<Config Name="PUBLIC_URL" Target="PUBLIC_URL" Default="https://audiobooks.example.com" Mode="" Description="Public URL if accessing from outside localhost. Required for OAuth." Type="Variable" Display="always" Required="false" Mask="false"/>